Randomizing item rankings using a randomisation list across multiple questions and participants | XM Community
Skip to main content

Hi All,
I have 9 items to choose from, and 2 ranking questions for each participant. In each question, I have 3 out of the 9 items for participants to rank.
Say I have a randomization list as below:
image.pngSo for every participant, you will go down a list accordingly.
For example, for participant 1, 1st question, it will be (item 3, 5, 2), 2nd question will be (item 4, 1, 9). For participant 2, 1st question, it will be (item 2, 8, 6), 2nd question will be (item 8, 7, 5). For participant 3, you will then go back to the top of the randomization list and start the process all over again.

How would I achieve this using the embedded data?

You can use a hidden question and from that question just carry forward 4 of the choices randomly and even present and loop this block the times you need. But those items will be randomly picked and you won't have control over it.
Let me know if this helps!


Hi grahulp5, thank you for your suggestion. Although that works and it is nice, unfortunately my supervisor wants it that particular order.


chingweelim
How will you determine what was shown to the last respondent. If you can ignore that option what you can do here is include a random number generator from 1-3 for each person and include the branch logic above the block of questions to determine the embedded data to be piped in the rank order question.
It would look like this:
image.pngThen you can pipe in within rank order question and repeat process for second rank order question:
image.pngHope it helps!


Deepak
I think what you are presenting is still in random order.
What I want is to follow the exact order of the randomisation list in my fist post.
image.png
There could be 3, 10, or 100 participants in total. So say if there are 4 participants, participant 1 will only answer the first two ranking questions (item 3, 5, 2) (item 4, 1, 9) in this particular order. participant 2 will then only answer the last two ranking questions (item 2, 8, 6) (item 8, 7, 5) in this particular order. participant 3 will then go back to the start of the list, and only answer the first two ranking questions (item 3, 5, 2) (item 4, 1, 9). Finally, participant 4 will then only answer the last two ranking questions (item 2, 8, 6) (item 8, 7, 5) in this particular order.
To simplify:
participant 1: (item 3, 5, 2) (item 4, 1, 9)
participant 2: (item 2, 8, 6) (item 8, 7, 5)
participant 3 (goes back to top of list): (item 3, 5, 2) (item 4, 1, 9)
participant 4: (item 2, 8, 6) (item 8, 7, 5)


chingweelim
The issue here is two participants can fill in at the same time and it would be difficult in the middle of the survey to track what is shown to the previous respondent. I am not sure how will you track it realtime.
Hence, a random order is advisable, hope someone else has a solution!


You will need to create 3 hidden screens to pick the variable in the order defined above and then pipe those into embedded and use that embedded for ranking question. This is how we create maxdiff setup.
To fetch the variable you can either use SDC (Supplemental data) or authenticator for a respective respondent and get the required choice number and then use JS to punch it via setChoiceValue to punch them.


grahulp5
Okay, I will look into SDC, authenticator and JS. Forgive me but I am very new to Qualtrics.

grahulp5 Deepak
In the meantime, I have been experimenting different ideas from your suggestions so far and found a rather crude solution. May not work for certain cases but at least it seems to work for my case.
Step 1:
I create a RespondentCount under Quota section (will come back to this later).
image.png
Step 2:
Next, I create Item block (Items 1-9):
image.png
Step 3:
Next I create two blocks. The first block is for Rank questions 1-2, the second block is for questions 3-4.
image.pngimage.png
Step 4:
In the survey flow, setup 'respodentNumber' embedded data. This is used to track the count of the participant, starting from 0 (not starting from 1 but we can work around it with simple math later).
image.png
Step 5:
Next, I will create two custom embedded data. the 'currentLastQuestion' modulo calculation determines the last question for the current user based on the respondentNumber (as mentioned, respodentNumber will start with 0, so I adjusted it to start from 1 here). So, (1*2) % 4 is 2, which is the last question for the first respondent.
So:
For responder 2, it will be (2*2) % 4, which the 'currentLastQuestion' becomes 0 (but I forced it to become 4 if that happens.)
For responder 3 (this is where you go back to the top of the randomisation list), it will be (3*2) % 4, which the 'currentLastQuestion' goes back to 2.

Then, for subsequent responders, I set a 'previousLastQuestion'. That is, the previous responder's last question.
image.png
Step 6:
Just below Step 5, I added additional adjustments.
If it is the first respondent (respodentNumber = 0), I will set currentLastQuestion=2 and previousLastQuestion=1.
Then, if currentLastQuestion=0 [when (2*2) % 4 occurs as I mentioned in Step 5], I set it to 4.
Finally, if previousLastQuestion=0, I set it to 1.
image.png
Step 7:
Finally, I set up the branch logic to show the desired order of the questions.
image.png

Some limitations:

  1. this only works in multiples of 2 for this case. If say there are 5 rank questions instead of 4, this would not work.

  2. Deepak mentioned that this could go wrong if two participants are taking this survey at the same time.

Again, I have to stress that this is a very crude method. I have not even seen what the dataset will even look like. Its a start, but I may need further tuning.


Leave a Reply