Randomized experiment | XM Community
Skip to main content
Solved

Randomized experiment

  • 15 July 2024
  • 4 replies
  • 25 views

Hello,

I am creating a randomized experiment. Two videos in the experiment need to be randomized. One is a control condition video and the other one is a treatment condition video. I did that and I have no issues with the video randomization. Now I have survey questions following the videos. Ideally, the survey questions need not be randomized and participants in both groups (control vs. treatment groups) answer the same questions. However, when I will analyze, I will compare the data between two groups. Now I am not sure if the survey questions following each video need to be built separately under each video or if survey questions can just be created in a separate block. 

What would be helpful when I will analyze data between two groups?

4 replies

nryalls
Rank
Userlevel 2
Badge +11

Hello, @Haimanti. My suggestion would be to use embedded data to note which video they saw to help in evaluating the results for the follow-up questions. It makes sense to me to have the follow up questions be one block as you can filter the results based on which video they saw. 

Below is a rough visual. If you have a randomizer to select between the two videos, then have a branch and select the question name for the control video for the If statement and choose “is displayed” then add an Set Embedded data element and name the embedded field and choose a value, here I used “Control.” You could add another branch logic to say if treatment video is displayed and set the embedded data video to “Treatment”. 

Then you would add the follow-up question block as a new element on the same level as the branch. This will then include in embedded data which video the participants saw. You can then filter the results between the two. You should also be able to use the embedded data as a data set in widgets like bar charts. 

 

 

Hope this helps! 

H
Rank
Badge +1

@nryalls Thanks for your reply. While waiting for a reply, this is what I did with my survey builder. I added the set of questions to each block and then I randomized. This is how my survey flow looks:

I am doing this for the first time. Do you this will work?

Thanks!

Haimanti

nryalls
Rank
Userlevel 2
Badge +11

Yeah, that will work - because they will see either block and it will include the questions you want them to answer. Depending on how you name your questions, I might suggest adding either “C” for control or “T” for Treatment to the beginning so that when you go to do analysis you know which Question 1 goes to which video. Does that make sense? 

H
Rank
Badge +1

@nryalls naming the questions is a great suggestion. Thanks a ton!

Best,

Haimanti

Leave a Reply


Terms & Conditions