We have encountered an issue with the group assignment randomization in our survey.

Our survey utilizes randomization tools to assign respondents to different treatments or groups. In one block of the survey, we included 20 questions, and hid them using Java Script. We employed Advanced Randomization to ensure that only one of these questions would be shown to each respondent, setting it to “Evenly display the questions”. We saved the code of the question that was displayed as an embedded data field to know which group respondents were assigned to (assignedGroup). Our goal was to assign respondents to different groups in an evenly distributed manner. (See screenshots below).

When previewing the survey, the randomization process worked correctly, as it does now. However, once distributed, all responses we received were from respondents assigned to the same group (group 20).This survey was a direct copy from our first pilot. In the initial pilot, the randomization was structured in the same manner, and it worked correctly when distributed via Prolific. Additionally, the Prolific link for this survey also worked correctly, as it redirected respondents to our Qualtrics survey without any issues.

We are unable to understand why this issue occurred, as the survey logic was tested and functioned correctly during the preview phase. If anyone has experienced similar issues or has expertise in survey logic and randomization, your guidance would be greatly appreciated. Additionally, we would appreciate any tips on how to prevent this from happening in the future. 

Thank you for your support!!