Randomized groups w embedded data - only one result shown | XM Community
Skip to main content

Hello everyone,

I have created a survey with two randomized groups (A/B) which each consist of fixed pairs of two items (A1/A2 and B3/B4). These item pairs have a standard question block including an embedded data field (“Brand”) and are shown in randomized order. Expected result: each participant sees two items (→ Brands) and I receive results for the standard question block for four brands in total.

Survey has already ended.

 

Basic problem:

Even as the preview runs worked fine and all testers were presented two brands in randomized order, I can’t differentiate the brands in the final survey results. More below. 

 

Background: Setup:

Randomizer / randomly present 1 of the following elements / evenly present elements

   → Group A:

         → randomizer  / randomly present 2 of the following elements / evenly present elements

              → A1 with embedded data field (Brand1), standard question block and a distinct attention check result ATT1, 

              → A2 with embedded data field (Brand2), standard question block and a distinct attention check result ATT2

→ Group B:

         → randomizer  / randomly present 2 of the following elements / evenly present elements

              → B3 with embedded data field (Brand3), standard question block and a distinct attention check result ATT2

              → B4 with embedded data field (Brand4), standard question block and a distinct attention check result ATT1

 

Embedded data was set to variable type: text

 

Detailed problem:

In e.g. SPSS I see the different results for the two standard questions blocks, but only one variable “Brand” for each participant. Hence I a) can’t assign the brand to the correct results and b) am missing a second “Brand” variable. I also don’t know if the displayed brand is connected to the first or second block of results.

Also when looking at individual responses in Qualtrics’ Data & Analysis tab, I always see the same single brand displayed for the two different standard question blocks - even if the participant’s answers vary. It should show two different brands, as it still does when I preview the survey.

 

Questions:

  • Is there any option to fix this in Qualtrics before exporting to SPSS? And avoid doing it manually?
  • Do you think that the data was presented correctly?
  • Is there any setting (e.g. “condition identifier”) which I should adapt for the next time?

 

Thank you very much for your help in advance! I tried to find answers in the other threads but could not connect those to my problem.

Can you share a screenshot of your survey flow, specifically how you assigned the variable brand and the randomizer?

Sure! As it will be to big for the full set up, here is the part for one of the two brand pairs.
 

 

You've overwritten your Hotel Brand and Logo variables, so you'll only see one value.

Your questions were most likely presented properly. 

You could try exporting the display order to see when the questions were presented. However, I doubt that will help. 

You've overwritten your Hotel Brand and Logo variables, so you'll only see one value.

Your questions were most likely presented properly. 

You could try exporting the display order to see when the questions were presented. However, I doubt that will help. 

Thank you. How can I prevent the variables from being overwritten?

 

The first principle is to preview both your survey and the data that it collects. You can generate automated responses (Click on Tools) to ease that, but nothing is better than manually going through the survey and then downloading the data to see if it matches your expectations.

In this particular, I am assuming that you are using the brand name in your questions, therefore, I would add another ED block here:

 

Let’s call it call it Display Order and set its value to itself  followed by the hotel brand.

${e://Field/Display%20Order}_${e://Field/Hotel%20Brand}

 

As display order will be initally empty, you’ll get values like _Hilton_La Quinta. The idea is to capture the order in which the brands were displayed so that you can map things accordingly.

This will need to be in a separate ED block and if you use the existing Hotel Brand ED block it won’t capture the most recent value.

Another way to do this would be to use branch logic. Add a branch here:

 

The idea here is that since you are entering the randomizer with an empty Hotel Brand ED, so lets set the value of the FIRST ED as the group we are entering. If however, the Hotel Brand already has a value, then that should overwrite the FIRST ED as that brand was already displayed.  

 

 

Also, if you only have one “Satisfaction” block and are reusing it for both brands, i.e., presenting it twice to the respondents, you will get only one answer value, the last one. This was a mistake that I repeatedly made in my earlier surveys.

Copy the block in your survey editor and use two different blocks in the survey flow.

 

Hope this helps.

 

Thank you very much for your help - also on the point with the single satisfaction block! This is indeed something I was not able to fix manually.

Leave a Reply


Terms & Conditions