Hi,
We are trying to add some math into our System Usability Scale (SUS) survey that will automatically calculate the score in the survey flow and reporting. We are trying to leverage the following article as a reference:
https://medium.com/@bobbyg/the-suspense-is-killing-me-3dac7eeb2ac
The challenge is that the Qualtrics interface has been updated and some of the instructions no longer align with the new interface. Any help would be greatly appreciated.
Bryan
Solved
System Usability Scale (SUS) Math in Survey Flow
Best answer by LaurenK
Hey @FCSA_UX! Check out this page about scoring! This page may also be helpful to get you started with your survey flow setup based on scoring!
View original
+27Can you be more specific? Are you having trouble with the calculation or reporting? Where don't the instructions align with the interface?
+13@FCSA_UX Hello there! I am having trouble getting the SUS formula to work in Qualtrics. Can you tell me what you did to get your to finally work? Thank you!!
Good morning all,
I faced the exact same issue. I used the medium article but the math syntax is also failing in my case. How did you make it work?
I faced the exact same issue. I used the medium article but the math syntax is also failing in my case. How did you make it work?
+6@Papada92 - Can you send your math operation? Often times it is a simple formatting issue. I am happy to give it a once over for you.
!
That would be brilliant, thanks! Tried putting spaces, removing the spaces but I can't seem to get it running.
Let me know what do you think is the issue @Akdashboard.
That would be brilliant, thanks! Tried putting spaces, removing the spaces but I can't seem to get it running.
Let me know what do you think is the issue @Akdashboard.
+6Hey @Papada92,
Yeah, your issue is just simple formatting. It is a lot to type out from just an image, so I am going to explain what you need to do. For extra help, reference math operations in Q Support (link)
So you DO need spaces. Basically you need a space after every unique item (i.e. variable + - * / ( ) etc.)
For example: X=(1+(2*3))/4 would be written X= ( 1 + ( 2 * 3 ) ) / 4.
When you are doing this with piped text you drop the {}, like you did and treat the rest of the statement as a unique item.
For example, this would be the formatting for the first little bit of your formula: $e{ ( ( q://QID1/SelectedChoices/Recode - 1 ) + ( 5 - q://QID1/SelectedChoicesRecode ) *etc.
Yeah, your issue is just simple formatting. It is a lot to type out from just an image, so I am going to explain what you need to do. For extra help, reference math operations in Q Support (link)
So you DO need spaces. Basically you need a space after every unique item (i.e. variable + - * / ( ) etc.)
For example: X=(1+(2*3))/4 would be written X= ( 1 + ( 2 * 3 ) ) / 4.
When you are doing this with piped text you drop the {}, like you did and treat the rest of the statement as a unique item.
For example, this would be the formatting for the first little bit of your formula: $e{ ( ( q://QID1/SelectedChoices/Recode - 1 ) + ( 5 - q://QID1/SelectedChoicesRecode ) *etc.
Hey @Akdashboard
I switched into this but still doesn't work:
$e { ( ( q://QID1/SelectedChoicesRecode - 1) + ( 5 - q://QID2/SelectedChoicesRecode ) + ( q://QID3/SelectedChoicesRecode - 1 ) + ( 5 - q://QID4/SelectedChoicesRecode ) + ( q://QID5/SelectedChoicesRecode - 1 ) + ( 5 - q://QID6/SelectedChoicesRecode ) + ( q://QID7/SelectedChoicesRecode - 1 ) + ( 5 - q://QID8/SelectedChoicesRecode ) + ( q://QID9/SelectedChoicesRecode - 1 ) + ( 5 - q://QID10/SelectedChoicesRecode ) ) * 2.5 }
I switched into this but still doesn't work:
$e { ( ( q://QID1/SelectedChoicesRecode - 1) + ( 5 - q://QID2/SelectedChoicesRecode ) + ( q://QID3/SelectedChoicesRecode - 1 ) + ( 5 - q://QID4/SelectedChoicesRecode ) + ( q://QID5/SelectedChoicesRecode - 1 ) + ( 5 - q://QID6/SelectedChoicesRecode ) + ( q://QID7/SelectedChoicesRecode - 1 ) + ( 5 - q://QID8/SelectedChoicesRecode ) + ( q://QID9/SelectedChoicesRecode - 1 ) + ( 5 - q://QID10/SelectedChoicesRecode ) ) * 2.5 }
+6Can you send me a screen shot of your Survey Flow?
+27Missing a space after the first "- 1", and an extra space after "$e". Corrected:
```
$e{ ( ( q://QID1/SelectedChoicesRecode - 1 ) + ( 5 - q://QID2/SelectedChoicesRecode ) + ( q://QID3/SelectedChoicesRecode - 1 ) + ( 5 - q://QID4/SelectedChoicesRecode ) + ( q://QID5/SelectedChoicesRecode - 1 ) + ( 5 - q://QID6/SelectedChoicesRecode ) + ( q://QID7/SelectedChoicesRecode - 1 ) + ( 5 - q://QID8/SelectedChoicesRecode ) + ( q://QID9/SelectedChoicesRecode - 1 ) + ( 5 - q://QID10/SelectedChoicesRecode ) ) * 2.5 }
```
```
$e{ ( ( q://QID1/SelectedChoicesRecode - 1 ) + ( 5 - q://QID2/SelectedChoicesRecode ) + ( q://QID3/SelectedChoicesRecode - 1 ) + ( 5 - q://QID4/SelectedChoicesRecode ) + ( q://QID5/SelectedChoicesRecode - 1 ) + ( 5 - q://QID6/SelectedChoicesRecode ) + ( q://QID7/SelectedChoicesRecode - 1 ) + ( 5 - q://QID8/SelectedChoicesRecode ) + ( q://QID9/SelectedChoicesRecode - 1 ) + ( 5 - q://QID10/SelectedChoicesRecode ) ) * 2.5 }
```
+6Good catch by @TomG.
The only other thing to check for is to make sure that your embedded data block with your calculation needs to be under the question blocks that house the questions.
The only other thing to check for is to make sure that your embedded data block with your calculation needs to be under the question blocks that house the questions.
Hello,
I've been trying to make the calculation work. To check it's working correctly I entered scores which should result in a mean of 85 according to my excel evaluation sheet. But no matter what I enter, the mean value is always 50.
I have been wondering, if Akdashboard 's tip about the embedded data block might be an issue for me? I don't really know what is meant by "it should be under the question blocks that house the questions". Maybe a screenshot would help?
Just in case there is a mistake there, I am adding my calculation, too:
Actually, there wasn't an option "SelectedChoicesRecode" available - only "SelectedChoices". I tried this with and without adding "Recode". (Recoding didn't seem necessary - I guess it's a default in the current qualtrics version?)
Many thanks for any tips you might have!
+27CharlotteL ,
I think you are always getting 50 because your piped fields are wrong:
-1 + 5 = 4 * 5 = 20 * 2.5 = 50
Actually, there wasn't an option "SelectedChoicesRecode" available - only "SelectedChoices". I tried this with and without adding "Recode". (Recoding didn't seem necessary - I guess it's a default in the current qualtrics version?)
Whether or not there is a SelectedChoiceRecode depends on the type of question. I'm guessing your questions are text entry, so there is no recode.
I recommend using the Piped Text choice from the drop down menu to make sure you get the correct piped string.
How would I need to change the setting to actually get offered to use "SelectedChoicesRecode"?+27You need to scroll down further and find "Ich würde..." (recode). The pipe will be something like ${q://QIDXX/SelectedAnswerRecode/1}.
+1Well, I'm stumped so I must be missing something obvious. I haven't seen this error happen to anyone else. I'm attaching a screen shot of my flow. No matter how many responses there are to the survey, I still see Count and Mean as 0 in the Results. I've re-added my piped data about a dozen times and I don't think anything is wrong there even though the names are a bit wonky (52? 53?). But really, it could be something to do with that. Anyone spot the problem? 
Pasting my code in here too, so nobody has to squint:
$e{ ( ( q://QID1/SelectedChoicesRecode - 1 ) + ( 5 - q://QID52/SelectedChoicesRecode} + ( q://QID53/SelectedChoicesRecode - 1 ) + ( 5 - q://QID54/SelectedChoicesRecode) + ( q://QID55/SelectedChoicesRecode - 1 ) + ( 5 - q://QID56/SelectedChoicesRecode) + ( q://QID57/SelectedChoicesRecode - 1 ) + ( 5 - q://QID58/SelectedChoicesRecode) + ( q://QID59/SelectedChoicesRecode - 1 ) + ( 5 - q://QID60/SelectedChoicesRecode ) ) * 2.5 }
+27Try this:
$e{ ( ( q://QID1/SelectedChoicesRecode - 1 ) + ( 5 - q://QID52/SelectedChoicesRecode ) + ( q://QID53/SelectedChoicesRecode - 1 ) + ( 5 - q://QID54/SelectedChoicesRecode ) + ( q://QID55/SelectedChoicesRecode - 1 ) + ( 5 - q://QID56/SelectedChoicesRecode ) + ( q://QID57/SelectedChoicesRecode - 1 ) + ( 5 - q://QID58/SelectedChoicesRecode ) + ( q://QID59/SelectedChoicesRecode - 1 ) + ( 5 - q://QID60/SelectedChoicesRecode ) ) * 2.5 }Changed a } to a ) and added spaces before a few )'s
+1Thank you so much for catching those typos. I was sure that would fix my problem but It seems I may have more than one problem, as I'm still getting the Count, mean, etc. displaying as 0 on the results page.
The individual question response data is displaying accurately on the results page below my custom SUS Score page.
I have 16 recorded responses so no idea why my Results page calculation is still displaying all zeros. I have double-checked my re-coded values for the questions, but I'm stumped.
Thanks! You seem to be the hero of this page.
+1I wonder if Qualtrics Support can help with this. I am seriously stumped and feel like I've tried everything since July.
Late to the conversation, but here is the solution I came up with after failing to get the equation to work in the embedded data.
First, I simplified the task by doing the math in the recode values rather than in the formula. So rather than recoding each question 1 - 5, I recoded odd items from 0 to 4 and even items from 4 to 0.
The resulting equation looked like this:
$e{ ( q://QID1/SelectedChoicesRecode + q://QID2/SelectedChoicesRecode + q://QID3/SelectedChoicesRecode + q://QID4/SelectedChoicesRecode + q://QID5/SelectedChoicesRecode + q://QID6/SelectedChoicesRecode + q://QID7/SelectedChoicesRecode + q://QID8/SelectedChoicesRecode + q://QID9/SelectedChoicesRecode + q://QID10/SelectedChoicesRecode ) * 2.5 }
Rather than throwing the equation in embedded data, which consistently would not play nice, I added it as a default response in a text-response question in my survey on a new page.
To keep this question from appearing for my participants, I used the javascript
Qualtrics.SurveyEngine.addOnload(function() {
jQuery("#"+this.questionId).hide();
this.clickNextButton();
});
Question for you, did you have your SUS set up as a Matrix Table or Multiple Choice questions? Mine is a matrix and it seems I can only enter recode values at the label level, which would mean every question would have the same values and that wouldn’t be correct for scoring.
Thanks,
Molly
Leave a Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.