Forget to change the number in " Randomly present xx of the following elements" in Randomizer | XM Community
Skip to main content
Question

Forget to change the number in " Randomly present xx of the following elements" in Randomizer

  • December 15, 2020
  • 4 replies
  • 39 views
J

In randomizer, I wanted to show only 1 element out of 80 elements. Those 30 elements are embeded data that determin which block they will be directed to in the future.
However, I forgot to change the number in " Randomly present xx of the following elements" from 80 to 1 when I added elements and already collected the data.
In the data, I got the embeded data in the randomizer for each subject. However, I am not sure whether it is correct. I tried with my survey myself, it seems that the mistake I made doesn't matter. Each subject still get one of the elements from randomizer. Only thing that matter is the edit count.
Is my understanding correct?
Thank you!
Capture.PNG

    Abhishek_U

    4 replies

    A
    Level 7 ●●●●●●●
    Forum|alt.badge.img+22
    • ahmedA
    • Level 7 ●●●●●●●
    • December 15, 2020

    Your question can not be answered without looking at the branch logic after setting the embedded variables. What can be said from the screenshot is that all participants would have potentially seen all the questions AND all of them would most likely have the same embedded variables. Which would imply that everyone saw the same stimuli and got the same questions. Just the order was changed.
    However, as mentioned earlier, the result maybe different based on your branch logic.

    J
    • JieruiYang
    • Author
    • December 16, 2020

    Hi ahmedA,
    Thank you very much
    https://www.qualtrics.com/community/discussion/comment/32680#Comment_32680for your answer! I have tried my survey and it seems that it works completely fine even if I forgot to change the number in randomizer. Each of the respondent get only one randomly assigned embeded variable and go to the branch that correspond to the assigned embeded variable.
    Below are the sreenshots of the branch logic.
    Capture.PNGCapture2.PNGCapture3.PNGCapture4.PNG

    A
    Level 7 ●●●●●●●
    Forum|alt.badge.img+22
    • ahmedA
    • Level 7 ●●●●●●●
    • December 16, 2020

    Okay. From what I can see, the data doesn't appear to be reliable. Here why I think that is the case.
    Let's say you have 4 elements in your randomizer. Two of them are setting embedded variables (E1 and E2) and the other two are questions (Q1 and Q2). You want the participant to see Q1 if E1 is set and Q2 if E2 is set. (If this assumption is invalid, then the explanation below is irrelevant.)
    Since the randomizer is set up to show all elements. All the 4 elements are going to be activated for everyone, only the sequence will be different. Here are 5 sequences out of the various possibilities.

    1. E1, E2, Q1, Q2

    2. E2, E1, Q1, Q2

    3. E1, Q1, Q2, E2

    4. E1, Q1, E2, Q2

    5. Q1, Q2, E1, E2

    Case 1, is fine. First E1 variables are set, then E2. Q1 is presented, fails due to branch logic, then Q2 is presented, and the person answers them. Similarly for case 2.
    In case 3, First E1 is set, they participant sees Q1, then Q2 gets activated, but fails due to branch logic, then E2 is set. So you are getting results for Q1 and the embedded variables for E2.
    Similarly, in case 4, first E1 is set, they answer Q1, then E2 is set and they answer Q2. So you are getting responses for both Q1 and Q2, but only see the embedded variables as E2. You don't see this in the data maybe that's because of the number of responses you have recorded. For 80 elements, you have 80! sequences.
    In case 5, you may have the participant seeing either both or one or none of the questions, but you are still getting the embedded variables.

    So you see, there is significant probability that your data is not reflecting the actual survey. It is possible that the data is fine, but I wouldn't make that conclusion without looking very carefully at the data and tallying everything.

    J
    • JieruiYang
    • Author
    • December 17, 2020

    Hi ahmedA,
    Thank you very much for your thorough answer!
    I now understand deeply how the randomizer works from your answers!
    Good thing is that from your answer I know that the data that I collected can reflect the actual survey because in my randomizer, I only included embeded data elements. Say, I have 4 embeded data elements in the randomizer, E1, E2, E3 and E4. I want them to see question 1, 2, 3 and 4 correspondingly.
    I add branch logic and Q1, Q2, Q3 and Q4 after the randomizer.
    So, if the sequence is E1, E2, E3, E4, then E4 is recorded and they see question Q4; If the sequence is E3, E4, E2, E1, then E1 is recorded and they see question Q1.
    Thank you again! I can rely on the data now.

    Terms & ConditionsAccessibility statement