Variable Rules CHANGED retroactively with no solution for fix
At some point in the past few months, Qualtrics engineering changed how variables worked. They didn't both to even tell those in their support department, let alone their customers. So, you all may be aware, here is the info. If a question used in a variable was skipped or opted out, the variable now adds in a 0. Prior to this change, a blank was left and no calculation would be made. For example, if you wanted to get a simple mean across 5 questions you would use a variable Mean=(a+b+c+d+e)/5, if a survey contained answers to all five questions the variable would be populated with the mean. If one or more of the questions was opted out or not answered the variable would be left blank. Thus, only true, valid means were calculated. NOW, in either case a 0 is added in, which completely distorts a rating to the negative. An opt out or no answer should not count as a 0, but it does! I have MANY such variables in MANY surveys, which are all now useless. The "solution" is to setup Scoring which gets piped into an embedded data field. However, it will calculate a weighted mean for ALL respondents, even if they opt out of 4 of the 5 questions. Thus, there is no way to make rating for only those who answered all questions in a set. In addition, it only works for future NEW survey data as the score data is not available in data and analysis nor in reporting. Therefore, this makes all historical data not useable, unless you export and calculate outside in excel. But, why setup scoring at all if you are going to export and calculate in excel??
Hi @Kathryn_Ann, thank you for making me aware of this. I think I may have found a solution to the issue. If you go into the scoring section and click on "scoring options" on the top right there is a checkbox that says "treat empty statistics for scoring categories as 0". Unchecking that box may fix the problem going forward. I don't know if that will fix the issue retroactively.
+7Hi everyone! Just wanted to jump in here and provide a bit of clarity. The option that @uhrxx005 suggested will work if set up initially before data is collected, but unfortunately will not apply retroactively.
However, our Support Team has put in a feature request for this to be possible. Therefore, if you would also like this functionality, if you have not yet already, we’d recommend posting this in our Product Ideas category, as it is a feature not already developed by our team. Be sure to review the How to Write a Good Product Idea template and our Ideation Guidelines before you post your idea!
@Emily, Why would Qualtrics think its okay to change something as significant as this without having a fix that works retroactively? Either Qualtrics knew this would break survey calculations and did it anyways, or they didn't do their research. Fortunately, I don't have any active surveys with variable calculations active at the moment, but if I had when Qualtrics made this change I would be very frustrated. This has likely broke a ton of active surveys and I would think this should be of the highest priority for Qualtrics to fix. It shouldn't require a dozen of us contacting them and requesting a fix.
+7We do value your feedback, @Kathryn_Ann and @uhrxx005, and can understand how frustrating this can be. We have personally passed along and expressed your concerns to our product engineering team so that they can take your situation into account. We apologize for the inconveniences that this has caused you during your data collection. For anyone else who would like to investigate if their survey data has been affected, this change went into effect on February 1st.
Leave a Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.