I am analyzing my data under the Data & Analysis tab. There is a filter called "Finished". I have How does Qualtrics define whether a respondent actually finished? Is this based on hitting the final next button?
The answer seems obvious but I'd like to confirm because I have mismatch between the number of respondents I got in Qualtics and the number of HITs I paid for in MTurk. There is a difference of 17 and I'm not understanding why.
Page 1 / 1
@MCR As far as I know "Finished" means that they have hit the final next or submit button, but depending on your survey settings, Qualtrics may or may not be collecting incomplete responses. The discrepancy is between the number of people clicking on your survey and the number of people completing it. It may also be worth checking with MTurk to see how they define a "hit" (in other words, is it an individual viewing the page where the link is, or clicking on the link and accessing the survey).
If it is 1 or True it is full complete, in case it is 0 or false it is partial complete or incomplete whose link is expired.
I would venture a guess that you have people getting screened out. Do you have a flow that "screens" people you don't want to take your survey? Did you set those parameters in MTurk?
I've never worked with MTurk before, but when I work with other panelists we make sure that I'm not paying for screened out individuals, assuming I'm really clear about who I need and what my screener looks like.
I've never worked with MTurk before, but when I work with other panelists we make sure that I'm not paying for screened out individuals, assuming I'm really clear about who I need and what my screener looks like.
Thank you @AdamK12 @Rock @C_Bohn @Kate for your responses. Yes @Kate , I do have people getting screened out. In fact, hundreds of people got screened out. Those parameters were set in Qualtrics. @AdamK12 I will verify how MTurks defines a hit. The weird thing is that I had a quota of 114 and Qualtrics says it actually filled 113 of the quota which is close to the MTurk request of 114. Where things get tricky is when I filter my data to see who was NOT screened out, I only have 97 surveys. I should technically have 113 or 114 surveys that were NOT screened out. There's probably an issue somewhere with the screened out part. I hope this makes sense.
@MCR How long has your survey been open?
Another thing that might be happening: If someone starts a survey, and your settings are set to "record" the response after 2 weeks, they might be pushed to recorded and therefore you're getting incomplete responses. That feels like a hiccup that might bee happening for your "not screened out 97" filter.
Another thing that might be happening: If someone starts a survey, and your settings are set to "record" the response after 2 weeks, they might be pushed to recorded and therefore you're getting incomplete responses. That feels like a hiccup that might bee happening for your "not screened out 97" filter.
Hi @Kate Thanks for that thought. You're right. This could throw things off. However, there was a mismatch between the number of completed surveys and the number of MTurk HITs completed before the automatic record feature kicked in.
Leave a Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.