On average, yes. But it might come up heads 98 times. This feature helps guard against the unlikely, which becomes less important as the number of responses goes up.

Is using this feature still appropriate for randomizing participants in a controlled experiment though? I don't quite understand it, and I'm unsure whether I should use it or not.

https://www.qualtrics.com/community/discussion/comment/31170#Comment_31170Yes, you should use it. It will ensure that respondents are randomly assigned to your three conditions evenly instead of just randomly. It will reduce the variability in how many respondents are assigned to each condition. It reduces the need to over sample and makes statistical analysis better since you are less likely to run into issues of small sample sizes if you cut the data by other criteria.
It does not guarantee that you will have equal numbers of completes because people could abandon the survey after being assigned but before completing, but it improves the odds.

https://www.qualtrics.com/community/discussion/comment/31171#Comment_31171Great, thank you for your insights. I will use it then.

https://www.qualtrics.com/community/discussion/comment/31171#Comment_31171Thank you!
I'm still not fully convinced about what it means for the scientific rigor of an experiment.
Is there also a more profound discussion of this somewhere? E.g. how does the evening out work in qualtrics? What assumptions do I have to make to argue this is not causing any biases?

I doubt you'll find a more detailed explanation of these anywhere. Qualtrics isn't aimed at scientific studies, its more for business purposes.
From my small testing of the randomizer (3-4 experiments, 10-12 draws, 2-4 elements in the randomizer), here's the difference that I noticed.

• Without evenly present: Qualtrics does a random draw and presents one element to the respondent.


• With evenly present:


• Qualtrics checks the number of times each element has been presented, shows the one which has been presented least


• If more than one element has been presented the same number of times (lowest), it does a random draw to present one


• It does this till all elements have been presented an equal number of times

https://www.qualtrics.com/community/discussion/comment/34518#Comment_34518It works by keeping count of how many times it picks each item in the randomization. I can't speak specifically to the Qualtrics algorithm, but typically it is done by first shuffling the set of items, then sorting them by count. Once that's done, as many items as are needed are picked from the top of the list.