Pairwise Comparisons texts: How to?

Question

For research into the readability of texts, I want participants to compare multiple sets of two texts. They have to state which text is easier/harder to read. I have a database of 25 texts, all these text have to be compared to each other. Since it will be a lot of work to create all the sets by hand, I was wondering if such a thing is possible in Qualtrics.

I was thinking of putting all the texts separately in Qualtrics, choose: present 2 of the elements, followed by a randomizer which presents all of the elements evenly. However, I was wondering if there is an option to present all the possible sets evenly. Because if text 1 ... text 25 are all presented i.e. 10 times, it doesn't mean that the combination 1 - 2 and 1 - 25 are presented the same amount of times. Or does it?

It would be great if anyone could help me out!

TomG · Accepted Answer

I think you can get close enough (i.e. you'll end up with a fairly even distribution of pairs) by putting your 25 texts as choices in an MC question and using advanced randomization to randomly display 2 of the 25 and evenly present them. If you want to ask each respondent about multiple pairs, put your MC question in a loop & merge block and loop for as many times as you want.This is easy enough to set up, so you could quickly generate some test data then download it with the "viewing order data for randomized surveys" and look at the distribution of pairs.A Plan B could be to use the survey flow randomizer to evenly present all 300 pairs (24+25)/2, but it would be much more tedious to set up and the resulting data would be more difficult to work with because you would be piping the choices.

PeeyushBansal · Answer

You can do so by conjoint by making possible combinations and then presenting 2 out of these at a time.https://www.qualtrics.com/support/conjoint-project/getting-started-conjoints/getting-started-with-conjoint-projects/

Sign up

Login to the Community

Scanning file for viruses.

This file cannot be downloaded