Solved
How to ensure equal completed responses when using randomizer for groups?
I'm using randomizer to divide participants into groups. Since I would like an equal number of participants in each group, I'm checking the box titled "Present Elements Evenly". However, this does not solve my problem. My survey consists of certain compulsory questions. I would only like to consider a response to be complete if these compulsory questions have been answered. (I do not care about incomplete responses, i.e. those can be discarded). I want to ensure that I have an equal number of COMPLETED responses in each group. In other words, I don't just want to present my question blocks evenly, I need EVENLY COMPLETED responses in each group. Is there any way I can accomplish this?
I've tried searching a lot, but have not found a response that works yet. I would appreciate any insight you may have. Thank you immensely!
Best answer by Akdashboard
@Newbie - Disregard my 2nd comment in the earlier post. I don't think it would work the way I was thinking and even if it did, it wouldn't be the best solution for your situation.
Since you don't know what an appropriate quota would be, I recommend that you set an arbitrary number appropriate for the size of your study (you will have to just take a guess, but as you will read, it doesn't really matter what you chose).
Next, you may want to create an alert when a quota is full, so that you can monitor the balance of responses and reset and/or readjust quotas as appropriate. To do this, create _n_ branching blocks (where _n_ is the number of groups you have) at the end of you survey flow, before the Survey End block (if you have one), and nest an embedded data block under each. Create an embedded data field called "[1,2,3,etc.] Quota Met" and create a static value called true. Program the branching blocks to be "branch if in _n_ group (you are probably doing this with embedded data already, so it would read "if embedded data equals [group indicator] AND Quota _n_ is met". Now, for the survey that fills a given quota is completed, you have an indicator that that Quota is met.
The next thing you want to do is set an Action Alert to email you when "[1,2,3,etc] quota met is equal to true.
+6You could use the Quota function. Set multiple quotas for each group then set the quotas to only ticks up if a completed response completed for the given group. Then, under your randomizer, apply branch logic above the group assignment block(s) that reads "display if Quota X is not met".
Since you will inevitably get some groups to reach quota before others, this will automatically shift so that the under-reported groups receive more possibilities to complete until all groups reach quota.
Depending on your study, you may need to monitor this in order to reset the quotas when appropriate or use double tier quota logic (i.e. display if quota X is not met (large quota number), nesting to another branch logic that read "display if quota x is not met (smaller branch quota)." This will provide you some additional buffer time to not loose any potential responses.
Thank you for the detailed response!
Since I do not know in advance how many responses I will get, I am not sure what number to set the quota to. Are you suggesting I make a speculation for the quota number for each random category?
Secondly, could you kindly clarify how to set the quotas to only tick up if response is completed for a given group? In other words, how can I tie survey completion to a +1 for my quota while preventing partial responses from adding to the quota.
I would really appreciate any insight! Thank you very much!
@Akdashboard I now understand the first part of your response quite clearly I think! I'm currently using quotas.
The part I'm not fully comprehending is when you describe how to get "some additional buffer time to not loose any potential responses." I'm not sure what you mean here. Could you kindly clarify how getting buffer time would help ensure equal completion numbers in all randomized groups, and how to achieve it? Thank you immensely for your time and guidance!
+6@Newbie - Disregard my 2nd comment in the earlier post. I don't think it would work the way I was thinking and even if it did, it wouldn't be the best solution for your situation.
Since you don't know what an appropriate quota would be, I recommend that you set an arbitrary number appropriate for the size of your study (you will have to just take a guess, but as you will read, it doesn't really matter what you chose).
Next, you may want to create an alert when a quota is full, so that you can monitor the balance of responses and reset and/or readjust quotas as appropriate. To do this, create _n_ branching blocks (where _n_ is the number of groups you have) at the end of you survey flow, before the Survey End block (if you have one), and nest an embedded data block under each. Create an embedded data field called "[1,2,3,etc.] Quota Met" and create a static value called true. Program the branching blocks to be "branch if in _n_ group (you are probably doing this with embedded data already, so it would read "if embedded data equals [group indicator] AND Quota _n_ is met". Now, for the survey that fills a given quota is completed, you have an indicator that that Quota is met.
The next thing you want to do is set an Action Alert to email you when "[1,2,3,etc] quota met is equal to true.
@Akdashboard Thank you!
My current set up is very similar to this one here: https://www.qualtrics.com/community/discussion/1286/random-treatment-assignment-while-controlling-for-quota-javascript
But this one does not include one of the steps you have described i.e. "Create an embedded data field called "[1,2,3,etc.] Quota Met" and create a static value called true." Do you think I need this step? What is the significance of the static value?
I think I get the rest of it. Thank you!
+6You would use that value to trigger an email to you when the quota has been met.
Example: Send email IF 2 Quota Met IS true.
Email: Quota for group 2 is full, check distribution of responses to see if quotas need to be adjusted.
Additionally, I'm branching if quota has NOT been met. However, your branching condition is "if embedded data equals [group indicator] AND Quota n is met".
I'm attaching my flow below. Do you think I'm doing this incorrectly?!
@Akdashboard Oh I see. So what you described in your second reply was only the email alert part, not the actual flow of the survey (the randomization part)?
+6Right, at the very bottom you want both conditions, otherwise you will receive an email each time a survey is completed and the a quota is met, even if the response isn't for that group.
Think of it this way, the quota will continue to be met for a given group for each survey taken, regardless of which group is taking the survey. If the logic just checks if the quota is met, it will always trigger. So you need the 2nd script of logic to essentially check if this was the last response that filled the quota.
+6> @Newbie said:
> @Akdashboard Oh I see. So what you described in your second reply was only the email alert part, not the actual flow of the survey (the randomization part)?
>
Bingo!
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.