Seeking Assistance on Randomized Treatment Assignment in Survey Design | XM Community
Question

Seeking Assistance on Randomized Treatment Assignment in Survey Design

J
Badge +1

Hello Everyone, 

I am writing here because I am encountering some issues when creating my survey for my thesis. I am studying whether matching a chatbot personality to the user can yield to a greater acceptance. 

To do so, after asking their personality type, I would like to assign each participant randomly to different treatments. However, with the way I’m doing things, the participant goes through all these blocks one after the other. Here below is a screenshot of the Survey Flow. 

Is there a way to go in which the participant selects an answer from a questions and is allocated randomly to one of these following blocks once?

Your assistance upon this matter is highly appreciated.

Best regards, 

Jorge Ibanez

 

11 replies

J
Badge +1

Hello Ahmed, 

Thank you for your input, but now that I’ve collected the data, somehow, the treatment data does not seem to show properly. 

In my survey, each participant should be shown one condition each, but on the data it’s shown as they have undergone through 2 conditions. Moreover, all it looks like all the participants are reserved, while it should not be the case. A friend of mine did the survey and clicked on Extroverted and it does not appear. 

 

 


Looking forward to your reply,

 

Best,

A
Rank
Userlevel 7
Badge +21

 

In this setup all your values will be overwritten except the onces circled. So at the end of the day you will have only one value.

 

Also, please read the support pages on Piped text and check how the different options work before  launching the survey. QuestionText is your question text. ChoiceDescription is the text for the choice and not the choice made by the respondent. That will be selected choices.

 

Given your setup:

 

This condition is always FALSE. That's why all you see if reserved.

 

Even if a participant selected extroverted, they would be shown the options for reserved.

 

 

A
Rank
Userlevel 7
Badge +21

Here’s a sample for you to see how things should be setup.

 

A
Rank
Userlevel 7
Badge +21

In your current dataset, the only valid responses (based on what you’ve described) are those who have answered reserved.

To identify which question they have answered, download your data set and see which of the four blocks has answers in it. If none of them had any inputs, then you’ll have to view each response individually (from the data and analysis tab) and see which question was shown to the respondent.

J
Badge +1

Hello Ahmed, 

Thank you very much for the detailed answer. 

I implemented the change, but somehow I am still unable to track the treatment given to the participants. 

There are the latest attempts with my survey. The chatbot column still appears as NA, when a value should be appearing.


My survey flow looks like this now:
 

All the embedded data have different names, do you know why this might still be the case?

Thank you very much in advance for your patience. 

 

Best regards, 

 

J
Badge +1

I believe I was able to find an alternative with the export options. However, I am not able to find the flow ID displayed within the columns FL57 and FL 62 in my data. Values such as “BL_eCBZaAZvZsAv2fQ” do not appear on my survey flow, as can be seen below. Might you have an idea how I could discern that?

 

A
Rank
Userlevel 7
Badge +21

Change this to display only 1:

 

J
Badge +1

Hello @ahmedA,

 

Thanks for pointing that out!


I changed that in my survey flow, and weirdly enough, in my previews it works with only one block appearing, but with the published survey link, I still get those 4 blocks one after the other. 

Do you know any idea why and how we can resolve that?

Looking forward to your reply. 

Best, 

 

A
Rank
Userlevel 7
Badge +21

You must publish the survey again after making any changes for them to reflect in the survey link.

This is so that you can test/preview changes before making them live.

 

Also, if you are using the same browswer, clear the cache and cookies before trying the new published link.

J
Badge +1

Hello Ahmed, 

Thank you for that advice! 

I published the survey again and it worked out how I wanted it to be. 

The thing is now, after collecting the data, the specific routing undertook by the participant is not reflected in the data-analysis. Do you know what I can do in the workflow or in the builder so that I can trace the specific treatment that has been assigned to the participant in the data analysis?

Thank you in advance for your help. I truly appreciate it!

Best regards, 

 

Jorge 
 

 

A
Rank
Userlevel 7
Badge +21

One option would be put each of your blocks in groups. Each group will contain an ED block, for example Treatment = X and the relevant treatment block. And put these 4 groups in the randomizer.

 

The other, post data collection, method is to look at your responses. If the question was shown to them and they responded, you’ll have data, otherwise it will be empty. If you download your data as CSV, you’ll get a question export tag, which you can use to identify which question they have answered.

Leave a Reply


Terms & Conditions