Rank Order Question Type - Adding an option for Unknown/NA | XM Community
Skip to main content
Solved

Rank Order Question Type - Adding an option for Unknown/NA

  • August 9, 2021
  • 3 replies
  • 788 views
L

I'm trying to design a rank order question using the drag/drop format, but want my respondents to be able to put "unknown" if they don't know anything about the item (versus ranking it lower in their listing). Maybe this isn't good survey design.... but can it even be done?

Best answer by AdamK12

Hi LynneH -- this is a really good question. Based on your use case, another thing you can do is have your respondents use multiple choice buttons (not radio buttons, but tick boxes that let them choose multiple responses) and then you/the analyst can do the rank ordering on the back end by arranging each answer choice in reverse order of how many respondents chose them.
You can then include "unknown" as an answer choice (though I would recommend calling it "unsure" or "don't know" because that might be a more accurate reflection of how they perceive the answer choice).
Rank order questions can be challenging from an accessibility perspective because e-readers don't handle them well (and for visually impaired people, they are hard to answer) compared to a multiple choice question.
I hope that helps!

3 replies

AdamK12
Level 5 ●●●●●
Forum|alt.badge.img+68
  • AdamK12
  • Level 5 ●●●●●
  • Answer
  • August 9, 2021

Hi LynneH -- this is a really good question. Based on your use case, another thing you can do is have your respondents use multiple choice buttons (not radio buttons, but tick boxes that let them choose multiple responses) and then you/the analyst can do the rank ordering on the back end by arranging each answer choice in reverse order of how many respondents chose them.
You can then include "unknown" as an answer choice (though I would recommend calling it "unsure" or "don't know" because that might be a more accurate reflection of how they perceive the answer choice).
Rank order questions can be challenging from an accessibility perspective because e-readers don't handle them well (and for visually impaired people, they are hard to answer) compared to a multiple choice question.
I hope that helps!

Adam Korengold, NLM/SCI (he/him/his)
L
  • LynneH
  • Author
  • August 10, 2021

Thanks for the advice - super helpful. This is my first time using this tool, so that's good to know.
I have 22 items in my list, so I had thought that using the rank order drag/drop would reduce respondent burden... I'm also going to have a very small sample size. This is less of a "survey" and more of a "input gathering" exercise (regarding what initiatives our group is going to pursue), where I didn't want people to provide input on possible initiatives with no knowledge of them. But I think you're right, I probably need to change to a different question format.
Survey Monkey has the ability to do this, but I didn't want to pay $400 for something that I'd only use once!

AdamK12
Level 5 ●●●●●
Forum|alt.badge.img+68
  • AdamK12
  • Level 5 ●●●●●
  • August 10, 2021

LynneH That makes sense--if the goal is to reduce response burden, I would definitely go with the multiple choice buttons because your respondents might find the rank-order exercise to be confusing with more than twenty choices. (The "sweet spot" tends to be between 6 and 8, in my experience.) And I agree with not using Survey Monkey for such a narrow use case...
Best,
Adam

Adam Korengold, NLM/SCI (he/him/his)
Terms & ConditionsAccessibility statement