The "Evenly Present Elements" algorithm | XM Community
Skip to main content

I am running a survey experiment and willing to relax “completely random” treatment assignment in order to have a roughly even distribution across my treatments with smaller n

 

"Evenly Present Elements" does this nicely, but I could not find a description of the actual algorithm running behind this option. Does anyone have the documentation somewhere?

 

This would be super important for understanding treatment assignment and to be able to convincingly claim causality---especially when random assignment was relaxed to conditional random. (And I don’t think “I ticked the box on qualtrics” explanation is sufficient in most cases.)

Qualtrics keeps an internal count for each randomized item and the counts are incremented when the items are displayed. Display is on a least-fill random basis (The items are sorted primarily by item count and secondarily by a random number with the top n being displayed).


Leave a Reply