I stumbled into the following Qualtrics blog post today:
https://www.qualtrics.com/eng/building-a-platform-for-distribution-a-b-testing/
and I was wondering if this is available in Qualtrics today? The blog dates from 2018.
The blog is about A/B testing a survey (so NOT about sending a survey for A/B testing a commercial or product features like on https://www.qualtrics.com/support/survey-platform/common-use-cases-rc/a-b-testing/, but A/B testing the actual survey, e.g. what subject line gets higher response rates, if you use a question directly in your email or not, if the sequence of your questions has an impact e.g. start or end with demographics, ...).
In the blog there's also a part about seeing the perfomance of the initial email distribution vs a first, second reminder, but I can't seem to find this distinction in my system.
Or, was the feature that the blog describes never made publicly available?
Page 1 / 1
The idea here...as I read it...is to manually put people into your A or B group using embedded data at the distribution level (which you can now do) and then putting them in different lists. Once you have set that up, you can then do different things with each group/list. Different emails or reminder delays etc. Once the survey collection is complete, you can report on the embedded data to see which group performed better.
On the reminder reporting...I don't see how to do that either. You could maybe add embedded data in the URL on the reminder that would be unique to each message and report on that unless someone knows how to see which message sent.
Also, if you haven't already check out the respondent funnel (note it doesn't track reminders) but may be useful to you.
Also, thanks for sharing that post.. It was a good read!
On the reminder reporting...I don't see how to do that either. You could maybe add embedded data in the URL on the reminder that would be unique to each message and report on that unless someone knows how to see which message sent.
Also, if you haven't already check out the respondent funnel (note it doesn't track reminders) but may be useful to you.
Also, thanks for sharing that post.. It was a good read!
Hi @MattBroffman_Orlando ,
Yeah I read about that "using embedded data at the distribution level" in the blog, but I must be searching on the wrong keywords but I can't find on the support pages how that works?
Yeah I read about that "using embedded data at the distribution level" in the blog, but I must be searching on the wrong keywords but I can't find on the support pages how that works?
You can set them in the contact list: https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/standard-elements/embedded-data/#SettingValuesFromAContactList
Or with query strings in the URL of the survey: https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/standard-elements/passing-information-through-query-strings/
If you are doing a/b testing on distributions, I would set your groups into different directory lists and then use Panel ID to track them. You can then do different distributions for each list.
Leave a Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.