Hi Community,
I'm experiencing inconsistent results with the Calculate Metrics task in a Qualtrics Workflow and would like to understand if this is expected behavior or if there are any known limitations.
Use Case
I have a workflow that needs to determine whether all assigned reviewers have submitted their responses before proceeding to the next step.
For example:
- Project ABC is assigned to 5 contacts maintained in a list.
- Each respondent receives a survey invitation.
- The workflow uses a Calculate Metrics task to count the number of responses received for that specific project/assignment.
- When the count of completed responses with filters as specific question response is Approved equals the number of respondents, the workflow should continue.
Issue
The output of the Calculate Metrics task is not always accurate when the workflow runs. In some cases:
- Recent responses do not appear to be included in the count.
- The calculated value differs from the actual number of responses visible in the survey results.
- The same workflow may return different counts at different times.
Questions
- Is there any delay or latency between survey submission and data becoming available to the Calculate Metrics task?
- Does the task operate on a cached dataset rather than real-time survey data?
- Are there known limitations when using Calculate Metrics to count responses for workflow decision-making?
- What is the recommended approach for reliably checking whether all assigned respondents have completed a survey?
- Would using a Web Service Task and the Responses API be more reliable than Calculate Metrics in this scenario?
