Creating a pairwise comparison task for 300+ stimuli (45,000 combinations!) | XM Community
Skip to main content
I'm looking to use a Qualtrics survey to perform what will end up being a massive number of pairwise comparisons. I have 300 visual stimuli, I need to have participants rate the visual similarity between all possible pairs of those stimuli. That's roughly 45,000 pairs.



Each participant will rate roughly 300 or so of these pairs. 15 participants for every pair.



The issue is how to have Qualtrics generate the pairs in its interface (obviously, I don't want to do this manually). Is there a way to create a gallery and have Qualtrics draw from it in an ordered fashion, to form all possible combinations? Additionally: Can this be done in sets of 300?



Any help at all would be appreciated. I have little experience with coding or scripts but would be happy to learn.
The only way to keep track of things across responses in Qualtrics is quotas. Setting up 44,850 quotas is unrealistic. Therefore, you really need to use a web service and an external database to randomly pick 300 pairs for each respondent and keep track of how many times each pair has been used. For example, you could write your web service script in php and use MySQL as the database.

https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/advanced-elements/web-service/

@tdelmore What did you end up doing? I have a similar need and I’m interested in knowing how you solved this quandary. 

Leave a Reply


Terms & Conditions