Since we are new to Qualtrics, me and my colleagues are very interested in how experienced users approach this randomization problem:
In a within-subjects study design, participants have to rate all images of an image pool of 138 images. 60 images of that pool they have to rate in control condition, 78 images they have to rate in experimental condition under cognitive load. The pool consists of 46 negative valenced images, 46 neutral and 46 positive valenced images. The randomization of images has to ensure that in both conditions the proportion of valences is 1:1:1 (i.e. 20:20:20 in control, and 26:26:26 in experimental group). The order of all images in a group has to be randomized as well.
Each condition should take place in its own block, and the order of blocks should be randomized (so a participant either rates all 60 images in a sequence in the control condition and then rates all 78 images in control condition or vice versa).
My first question: is this kind of stratified randomization (sampling without replacement) between two conditions possible inside qualtrics without programming? Could you give the basic logic how to accomplish it?
My second question: if the answer to my first question is "no" or if one would like to exercise even more control over the randomization process (i.e. including more variables in stratification), is it a valid approach to externally generate randomized stimulus lists an then passing these lists by a web service + piped text for each participant into Qualtrics?
Thanks!
I am sure there is a more elegant way to accomplish this, but what comes to mind for me in order to accomplish stratified randomization (sampling without replacement) between two conditions as you describe would be a combination of randomization and a lot of embedded data. It would be cumbersome, with so many options, but the basic approach would be:
- Create a separate block for each image in the survey.
- In survey flow, create a "Group" corresponding to each block. The group includes the Block, and also an Embedded Data field that sets a value for that block (eg in the block for the first image, set embedded data Image1 = Control). If a block's group is seen, the corresponding embedded data value for that block is also set, so you have something to use to prevent it being seen again. (Technically you could just set "Image1 = Used" but I'm assuming you would want to be able to know which condition each image was part of.)
- Place all the "groups" in the "Control" randomizer and set how many of them should be seen.
- Next another randomizer, but this time there's branch logic: the group is only seen in the instance that its embedded data does not have a value (is empty). So if Image1=Control, it would not be presented as part of the randomization. If embedded data Image1 is blank, the group is seen, and this time you set the embedded data to capture the other condition (eg Image1= Experimental)
You could nest the randomizers inside each other to handle the different valences, and/or put a master randomizer that determines whether the participant sees "All Control" then "All Experimental" or the reverse. But there is a lot of survey flow manual programming and a massive amount of Embedded data to accomplish the above (but, no external-to-Qualtrics programming needed). Someone else may have a more elegant solution but just using the base platform tools, that is what came to mind for me.
CarolK Thank you for answering and wrapping your head around this! Your suggestion gave me a lot of inspiration and insight in the possibilities of randomizers, branching, nesting and embedding; for example, I didn't know the "technique" to mark blocks with an embedded field yet. The downside is a lot of overhead and complexity which makes the survey hard to maintain.
Leave a Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.