A/B testing a survey | XM Community
Skip to main content
Solved

A/B testing a survey

  • February 7, 2020
  • 3 replies
  • 140 views
JoycaV
QPN Level 4 ●●●●
Forum|alt.badge.img+27
  • JoycaV
  • QPN Level 4 ●●●●
I stumbled into the following Qualtrics blog post today: https://www.qualtrics.com/eng/building-a-platform-for-distribution-a-b-testing/ and I was wondering if this is available in Qualtrics today? The blog dates from 2018. The blog is about A/B testing a survey (so NOT about sending a survey for A/B testing a commercial or product features like on https://www.qualtrics.com/support/survey-platform/common-use-cases-rc/a-b-testing/, but A/B testing the actual survey, e.g. what subject line gets higher response rates, if you use a question directly in your email or not, if the sequence of your questions has an impact e.g. start or end with demographics, ...). In the blog there's also a part about seeing the perfomance of the initial email distribution vs a first, second reminder, but I can't seem to find this distinction in my system. Or, was the feature that the blog describes never made publicly available?

Best answer by MattBroffman_Orlando

You can set them in the contact list: https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/standard-elements/embedded-data/#SettingValuesFromAContactList

Or with query strings in the URL of the survey: https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/standard-elements/passing-information-through-query-strings/

If you are doing a/b testing on distributions, I would set your groups into different directory lists and then use Panel ID to track them. You can then do different distributions for each list.

    3 replies

    MattBroffman_Orlando
    Level 2 ●●
    Forum|alt.badge.img+4
    The idea here...as I read it...is to manually put people into your A or B group using embedded data at the distribution level (which you can now do) and then putting them in different lists. Once you have set that up, you can then do different things with each group/list. Different emails or reminder delays etc. Once the survey collection is complete, you can report on the embedded data to see which group performed better. On the reminder reporting...I don't see how to do that either. You could maybe add embedded data in the URL on the reminder that would be unique to each message and report on that unless someone knows how to see which message sent. Also, if you haven't already check out the respondent funnel (note it doesn't track reminders) but may be useful to you. Also, thanks for sharing that post.. It was a good read!
    JoycaV
    QPN Level 4 ●●●●
    Forum|alt.badge.img+27
    • JoycaV
    • Author
    • QPN Level 4 ●●●●
    • February 11, 2020
    Hi @MattBroffman_Orlando , Yeah I read about that "using embedded data at the distribution level" in the blog, but I must be searching on the wrong keywords but I can't find on the support pages how that works?
    MattBroffman_Orlando
    Level 2 ●●
    Forum|alt.badge.img+4

    You can set them in the contact list: https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/standard-elements/embedded-data/#SettingValuesFromAContactList

    Or with query strings in the URL of the survey: https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/standard-elements/passing-information-through-query-strings/

    If you are doing a/b testing on distributions, I would set your groups into different directory lists and then use Panel ID to track them. You can then do different distributions for each list.

    JoycaV
    Terms & ConditionsAccessibility statement