Grouping questions within loop and merge block for randomization | XM Community
Skip to main content

I have spent days reading support docs and threads trying to figure out how to do what I need, and I keep hitting a point where there’s a line like, “Qualtrics just doesn’t do that.” I’m hoping there’s something I’ve overlooked or maybe some JavaScript that could be added to make it work. Here’s what the study wants to do:

  • Start out with consent and a couple of screening questions.
  • Select a random 20 of 50 video segments that each feature two characters.  This part is easy enough.
  • Use something (I’ve been looking at loop and merge with randomization, survey flow with randomizer and groups, etc.) to show the same page of questions for each clip that has been selected. Each page would include the following questions (I’ve used LM fields to represent the data per clip, but would obviously add manually to each block if using survey flow instead):
    • Video clip embed code from LM field based on the randomized selection]
    • Multiple choice question, 3 answer choices spulled from LM fields in the line for that video clip]
    • kCharacter 1 image and name from LM fields] matrix question, 4 statements/5 answer choices
    • kCharacter 1 name from LM field] bipolar question
    • cCharacter 1 name from LM field] text entry question to provide context for the selected ratings
    • aCharacter 2 image and name from LM fields] matrix question, 4 statements/5 answer choices
    • bCharacter 2 name from LM field] bipolar question with 1 statement, 5 answers 
    • =Character 2 name from LM field] text entry question to provide context for the selected ratings
    • Bipolar question with 3 statements, 5 answers about the video clip in general
    • Multiple choice question about the video clip in general

Within that page, I would like to randomize which character is asked about first (to deal with question order bias) -- so some participants would see the blue questions first and others the yellow. There are multiple questions per character because I need to have an open text box for explanation after they choose their ratings in the matrix, and qualtrics still doesn’t have that feature despite years of people asking for it… I use JS to remove the lines between those questions so they look like one. The bipolar question in the middle could be modified to be another matrix statement if only having matrix+comment per character would make this possible. But the thing I’m running into is the separation of features.

  • If I use Loop & Merge, there would be no way to randomize character 1/character 2 order unless there was only a single question per character. There’s no “nested L&M” option, and advanced randomization on the block would allow me to keep the beginning and ending questions fixed and randomize the middle, but doesn’t allow any kind of grouping to keep things together, so I could wind up with something displaying like this (not okay) instead of in character question groups:
    • lCharacter 2 name from LM field] bipolar question with 1 statement, 5 answers 
    • iCharacter 1 image and name from LM fields] matrix question, 4 statements/5 answer choices
    • iCharacter 1 name from LM field] text entry question to provide context for the selected ratings
    • lCharacter 2 name from LM field] text entry question to provide context for the selected ratings 
    • >Character 2 image and name from LM fields] matrix question, 4 statements/5 answer choices
  • ​​​​​​​If I use survey flow with a randomizer, I can create a group for each video clip/question set, then have blocks for the intro questions, character 1, character 2, and the wrap-up video questions. A second level within that group could randomize between the two character groups. The would do what I want in terms of question order, but then it would force three page breaks  for each video clip, and I need the questions per clip to be on the same page for user experience and timing reasons. 
  • I could use alternately survey flow with a randomizer, have a group for each video clip, then next another randomizer and have it choose from a block where character 1 goes first for every video and , a block where character 2 goes first for each video. This would let me keep all the questions in one block per video clip but then it’s not actually random order (they would get 1, 1, 1, 1, 1 or 2, 2, 2, 2, 2 as first-displayed, vs. with randomization of character they could get something like 1, 2, 2, 1, 1, 2 (or other viariations) that would be better for the sake of reducing bias.

As an added bonus, I need to be able to record the display order of both the video (with associated questions) and the character order for each video. It seems like there may be a JS hack to do this using hidden questions/embedded data or something, but I need to get the overall flow working before I start tryingthe thigns I saw in other threads about working around the lack of exporting display order with the data. 

Is there any way to do this that would keep all questions on one page per randomized video clip, but would allow me to randomize which character’s questions displayed first? 

Any help greatly appreciated. 

If you use a loop, you could randomize the characters into embedded data fields that you pipe into the loop.  

Randomizer Show 1 of 2 Evenly presented
  Set Embedded Data
    CharacterA = Character 1 name
    CharacterAimg = Character 1 image
    CharacterB = Character 2 name
    CharacterBimg = Character 2 image
  Set Embedded Data
    CharacterA = Character 2 name
    CharacterAimg = Character 2 image
    CharacterB = Character 1 name
    CharacterBimg = Character 1 image

Then pipe CharacterA, CharacterAimg, CharacterB, and CharacterBimg into your loop & merge fields.  You’ll have to unwind the data on the back-end to understand what questions go with each character, but it should accomplish what you want from a presentation and randomization perspective.

Thanks for responding! I’m trying to parse how that would work. Part of it might be the A-B, 1-2 thing, so in my response instead of character 1 and 2, I’m calling them left and right (for where they appear in the video clip) and display first or display second.

 

Without randomization of characters, I have a regular loop & merge table that looks something like this:

Field 1 (ClipID)

Field 2 (CharL)

 Field 3 (CharR) Field 4 (Embed Link) Field 5 (CharL-img) Field 6 (CharR-img) Fields 7-9 (More fields...)
ATYP1 Sam Riley <div, iframe, etc> <img src= etc> <img src= etc> (more values...)
EGBO1 Matilda Genevieve <div, iframe, etc> <img src= etc> <img src= etc> (more values...)
EGBO2 Alex Nicholas <div, iframe, etc> <img src= etc> <img src= etc> (more values...)
(47 more rows of clip ids) (more CharLs) (more CharRs) (more embeds) (more ImgLs) (more ImgRs) (more values...)

 

and in my loop & merge block, I had it set up like this:

pBEGIN BLOCK]

Watch this 20-second video, then answer the questions that follow.

${lm://Field/4}

-------------------------

${lm://Field/2}
Rate this Character: ${lm://Field/2}  

blah blah ${lm://Field/2}

>bipolar question]

blah blah ${lm://Field/2} 

etext entry comment]

--------------------------

${lm://Field/3}
Rate this Character: ${lm://Field/3}  

blah blah ${lm://Field/3}

}bipolar question]

blah blah ${lm://Field/3} 

}text entry comment]

--------------------------

-bipolar question relating to video overall]

lmultiple choice question relating to video overall]

lEND BLOCK]

... meaning in this example, Sam, Matilda, and Alex would all show first on the pages for their video clips.

If I follow your example, I put the randomizer with 2 embedded data elements in the survey flow before the loop and merge block.

I tried it first by putting the character names and image urls into the values in the set embedded data elements, with the intent that I would replace the names and urls in the loop & merge table with piped text for the corresponding embedded data field. But since I have different characters in the various clips, not 50 clips that all have the same two characters, it seemed like it wouldn’t really work because it could mix up characters from differnt videos. Then I thought maybe I’d misunderstood you, and instead of meaning I should pipe the embedded label into the l&m table, you meant I should pipe the embedded label into the question and leave the l&m table alone. 

Doing that, I wound up with this:

cImage description: screenshot of survey flow screen showing elements in this order:

  • Consent block
  • Welcome/screening questions block
  • Randomizer 1 of 2 evenly presented
    • Set embedded data element, CharL displayed first
    • Set embedded data element, CharL displayed second 
    • Loop & Merge block using randomizer to show 20 of 50 rows in l&m table]

Then, instead of piping ${lm://Field/3} from the loop and merge table into the question area, I would pipe the embedded data label, ${e://Field/CharDisplayFirst} and based on the assigned randomization condition, it would pull either field 2 or field 3 from the loop and merge table. 

 

This keeps all the questions for a video on one page (hooray!), but seems like it’s not really randomizing per video (not hooray 😞 ) -- if the randomization picks which ordering it will use before the L&M block begins, it would use the same selected order for all 20 videos they watch, rather than randomizing who gets shown first with each video. So a participant would get 1, 1, 1, 1, 1, 1 or 2, 2, 2, 2, 2, 2 as a list of which character is dsplayed first with each video, but couldn’t get 1, 2, 2, 1,1, 2 or 2, 1, 2, 2, 1, 1 or another random set of fist vs. second order for a character as they cycle through the video clip loops. This would maybe reduce question order bias a little bit, but it wouldn’t really address it, because the character in a specific position would always be asked about first, creating a pattern of primacy that would introduce bias. It seems like it would basically be, “this participant will see 20 videos and the questions about the character on the left will always come first or second depending on which of the two embedded data groups they were put into,” rather than, “for each video a participant sees, it will be random which character’s questions appear, so sometimes they’ll answer questions about the character on the left first and other times they’ll answer about the character on the right first.”

Have I completely misunderstood your suggestion, or is that just as far as Qualtrics can go with randomizing grouped questions displayed on the same page, and actual randomization of display order for the character question set isn’t possible at all?

No, you didn’t misunderstand.  From your original message, I thought  you were referring to the same two characters across all the videos in which case it would be confusing to to randomly switch the order on a respondent.

Now that I have a better understanding I would take a different approach.  Since all the questions are on the same page, you could write JS to randomly move the CharR questions before the CharL questions about half the time.

Ah, thanks! Any chance you can point me in the direction of any threads or tutorials on how to do that? I’ve been doing searches to try and find something as a starting point all day, but none of the threads here or in the Q&A or tutorials I can find on the web outside of the Qualtrics site are trying to do the same thing, and of all the js examples I’ve found in threads, I haven’t seen any that move a few questions to a different location in the block sometimes (which was exactly what I was wondering was possible originally, so it’s cool that you say it is).

When I couldn’t find any references to something that would similar to “randomly move the CharR questions before the CharL questions about half the time” I started looking to see if there were any examples that instead created a variable from an array and randomly chose 1. Found one, but I’m not sure if it would actually work once I modified it enough to fit my kind of data. 

Any pointers or links greatly appreciated. 

@jinx,

You can get all the question divs on a page using var qs=jQuery(“.QuestionOuter”);. Then you can move them around.  For example, to move the second question before the first question qs.eq(0).before(qs.eq(1));.

I thought I’d follow up here in case anyone else comes across this thread trying to do the same thing. I had a friend help me and this is the js I have now. He created a hidden text entry question as the first in the loop and merge block, which is where we have the randomization order being recorded, the l&m table row to ID the clip, and which character shows first (flipped/not flipped) -- these are being recorded for use in analysis. 

Qualtrics.SurveyEngine.addOnload(function()
{
	// jQuery( this.questionContainer ).hide();

	// With the loop and merge script, there seems to be a different prefix based on which entry is being used.
	let match = this.questionId.match(/^(\d+_)/);
	let prefix = matcha0];

	let flipped = 'Not Flipped';
	if ( Math.floor( Math.random() * 2 ) == 0 ) {
		// Fifty percent chance.
		jQuery( '#' + prefix + 'QID14Separator, #' + prefix + 'QID14, ' +
			   '#' + prefix + 'QID13Separator, #' + prefix + 'QID13, ' +
			   '#' + prefix + 'QID12Separator, #' + prefix + 'QID12' ).insertAfter( '#' + prefix + 'QID17' );

		flipped = 'Flipped';
	}


	jQuery( this.questionContainer ).find('input').val( flipped + ' Prefix: ' + prefix +  'Order:' + '${lm://CurrentLoopNumber}');


});

 

Leave a Reply


Terms & Conditions