Norming Questionnaire and Counterbalancing Lists | XM Community
Skip to main content

Hello, 

 

I am running a norming study where individuals will be rating words and pictures based on three questions that are the same across each word or picture they view. To my limited knowledge, it seems like the ‘Loop & Merge’ feature will help me with not having to rewrite those same three questions over and over. However, now I have a question on the stimuli set. I do not want to embed 400 unique pictures and words into the survey and instead am hoping I can create a stimuli library where the survey can pull the stimuli that are presented from a library that I create.

  • Can this be done with the Embedded Data function?

The study shows stimuli based on two different variables with two levels: Stimuli Type (which is either Picture or Word) and Stimuli Content (General or Specific). This makes up four conditions, with each condition having 100 stimuli, for a total of 400 stimuli. There is no way for each participant to view and judge all 400 stimuli.

  • Is there a way to present an equally randomized sample of the 400 stimuli across the four conditions? For example, can I code it so each participant sees 50 equally randomized items from the four conditions pulled from the embedded data library?

 

Thank you!  

@jaySA 

do not want to embed 400 unique pictures and words into the survey and instead am hoping I can create a stimuli library where the survey can pull the stimuli that are presented from a library that I create.

  • Can this be done with the Embedded Data function?

The words/pictures can actually store on the loop & merge table, you will need 100 rows (for 100 stimuli) and 4 columns (for your content, picture should be saved as picture link from your own online storage or uploaded to Qualtrics library. You can present the picture through HTML code)

  • Is there a way to present an equally randomized sample of the 400 stimuli across the four conditions? For example, can I code it so each participant sees 50 equally randomized items from the four conditions pulled from the embedded data library?

You can present them randomly and set the number of loops by check “Randomize loop order” and how many loop you want to present

In each loop, the content will be presented though a set,e.g: Words & Pics 3 will go together, Words & Pics 1 will go together in next loop..etc..

If it’s what you want, hope this helps

Hello, 

 

Thank you so much @Nam Nguyen. So just to clarify: 

 

The ‘randomize loop order’ function could present only 35 of each of the four fields. How would I ensure that the randomization is presenting the stimuli in an equal fashion? Since this is a norming study, we would want subjects to see a set of stimuli, but each item is judged by an equal amount of people. Thus, person 1 might see a random set of 25 stimuli, but person 2, 3, and 4 will see another random set of 25 without crossover to ensure that all stimuli are eventually being judged in an equal fashion.

  1. Can this account for that?
  2. Does it matter if there are more stimuli in one field than another? Will having an unequal balance across the fields mess things up?
  3. Additionally, can you set different randomization numbers across the four fields? For example, if there are only 60 GenPic, can one person judge 30, but if there are 120 GenWords, can they see 40?

Any resources you have that explain this more too? Thank you! 

Hi @jaySA 

Loop & Merge is balance with a preset, so with your unbalance set, Loop & Merge is not for you, because you will have a lot of empty fields in your loops.

Randomization presents everything in an equal fashion by it’s nature. The question is: Do you have enough responses for it to reach the expected balance?

Leave a Reply


Terms & Conditions