Broke my data by using move up/down in matrix statement list in live survey | XM Community
Skip to main content

I was under the impression that the matrix statements that show as column headers in the exported data (Q21_1, Q21_2, Q21_3, etc) were tied to the statement text, but after a recent export it looks like it’s based on the current order they are listed. I hadn’t expected this, so I used move up/move down in the statements list to re-order the display of statement scales during a live survey a couple of weeks back. For example, at launch:

Q21_1 I like birds istrongly disagree to strongly agree]

Q21_2 I enjoy petting cats

Q21_3 I don’t like snakes

 

After move up/down change, I expected it to do this:

Q21_3 I don’t like snakes

Q21_1 I like birds 

Q21_2 I enjoy petting cats

I expected that because that’s what I’d seen doing a move up/down of answer choices in a multiple choice question (specifically in SPSS export, where an MC question that used choice groups gave a column to each answer choice and populated a value of 0/1 to each).

 

But in an export after the matrix statement move up/down the data shows this:

Q21_1 I don’t like snakes

Q21_2  I like birds

Q21_3  I enjoy petting cats 

It never occurred to me that other question types would use a different pattern (I recognize this was not a good assumption!). So obviously my incorrect assumption has caused a massive issue in my data, and I need to figure out how to fix it. By default, I’m guessing that I’ll need to check the version history for the timestamp of publication for the revision where I published the up/down change, then manually edit any responses that came in after that time by hand. Annoying, but not impossible. However, due to multiple loop and merge blocks using this scale, I’m going to have to do this a LOT of times. 

  • Is there any kind of label applied to a matrix statement that persists the way the label persists for an answer choice? I’m pretty sure I did this same thing when I added in some attention checks (select strongly agree for this line, etc) earlier, and I’m wondering if there’s any way to double check what a statement was at the time a participant answered. My guess is that there’s not and statements are treated more like question wording than answer choices (which makes sense in that they are extensions of the question, but doesn’t in that regular question text can’t be re-ordered since it’s just one thing), in which case I may wind up having to throw out data if I can’t find everything I need in the version history notes.
  • Is there a better way to fix my mistake that what I’m planning (find/replace for all responses that were started after the move up/down was published)?

Ugh, I hate that I didn’t think to check the export to confirm the behavior, but here I am. Any info/advice that might be useful greatly appreciated. Thanks!

Seems like instead of moving the statements,  you changed the recode values. 

See if  this can help you.

 https://www.qualtrics.com/support/survey-platform/survey-module/survey-tools/saving-and-restoring/

 

Check the time when you move the statements in version history and check the same time in the data.. This can help to some extent if you don’t have much responses. 

 

Also, please review this before making changes to the live survey.

https://www.qualtrics.com/support/survey-platform/survey-module/survey-tools/saving-and-restoring/


I didn’t recode values, but your response had me go looking in recode values to look around and I found a solution/something weirder…

I did actually use recode values on the matrix questions some time ago, but not for statements -- I used it to change the coding of the strongly disagree - strongly agree choices from 1-4 to 0-3. I know there’s no way to change the text of statements in the recode values popup, but after your response, I checked out the popup again. I looked to see what the Question Export Tags view said, since I’d never used that, and that’s where I found the solution/where it gets weird… 

In the Recode Values popup with the Question Export Tags box checked, it shows the behavior I was expecting all along:

In the exported data I already had open, it was this (real statements, no animal examples this time):

Q21_1 This person seems likeable

Q21_2 I would hang out with this person in my free time

Q21_3 This person seems attractive

Q21_4 This person seems honest

Q21_5 This person seems socially awkward (and so on)

 

I re-exported after checking the box, and now it is exporting what I expected:

_5 likeable, _7 hangout, etc. 

 

  1. What a relief! I can go in and check that box in all the scales that were changed and re-export instead of breaking up the export and manually comparing thousands of pieces of data against previous versions.
  2. How in the world is that not the default behavior, to keep the labels with the statements in the export instead of changing them?
  3. Thank you!! I didn’t actually recode them myself to create the error, and I’d glanced at the recode popup earlier but saw the uneditable statements on the main tab without looking at the export tag checkbox (based on the label I would have thought was something to let you add notes to the question as tags based on how tag is used elsewhere as nomenclature). 
  4. I also looked at my version history already but hadn’t noticed the restore version link up in the corner. So thank you for pointing to the article for that. 

Hooray! My data cleaning for this mistake just went from what I’m guessing would have been 10-12 hours to about 15 minutes. Thanks again. :)

 


Follow-up in case anyone comes across this later in a search: don’t get excited like I did! Unlike when you use recode values to change the values of the answer choices and the next export will recode the previous data in each cell to match the new structure, turning on the question export tag does not. It changes the headers in the new export to match, but it doesn’t shift the values around, so I’m going to have to spend the weekend with version history and copy paste to fix it after all. 


Cool! Its great you shared for others too


Leave a Reply