Hello
We've just run a survey where we embedded a survey question in the email for the first time.
Interested in knowing other people's experience of conversion rates to the rest of the survey by doing this.
The embedded question was the NPS question, once they answered this, the rest of the survey opens in a new tab. The second question in the survey was the usual open-ended question asking why they gave that rating.
Just 36% of those that answered the NPS question then completed the verbatim question.
In previous surveys, where we have included a survey link within an email the % that answered the verbatim question after the NPS question ranges from 65% to 100%!
Have others had similar experiences? Makes me question the value of embedding the survey question in the email, as we get so much value from the verbatim questions to help us understand their NPS ratings.
Thanks
What are conversion rates from survey questions embedded in an email (in line questions)?
Best answer by InessaG
There is no way to know, unfortunately. You will get false responses. The only way we found out is through closed loop process - we were calling customers to follow up on their "negative" feedback and they said they never even opened the email. Our security team advised us on how anti-spam tools worked. We remove inline questions from all our survey invitations after that. Now we just have an inflated "partial response" rate on our dashboards because anti-spam tools are clicking into the survey link and Qualtrics is registering that as a partial response, but at least we are not getting false responses in our data.
+19Is your audience B2B or B2C? If B2B, they have sophisticated spam filters testing your emails, "clicking" into your answers giving you false responses. We found this out the hard way.
Thanks for the response. It's B2B, so that might help explain it but is rather worrying. How did you find out which ones were false and which ones were real?
+19There is no way to know, unfortunately. You will get false responses. The only way we found out is through closed loop process - we were calling customers to follow up on their "negative" feedback and they said they never even opened the email. Our security team advised us on how anti-spam tools worked. We remove inline questions from all our survey invitations after that. Now we just have an inflated "partial response" rate on our dashboards because anti-spam tools are clicking into the survey link and Qualtrics is registering that as a partial response, but at least we are not getting false responses in our data.
Hi InessaG
I raised this with Qualtrics Support and they came back with the following:
In this case there is an option, the Survey options > Security >
Fraud DetectionSecurity Scan Monitor This will Prevent security scanners from accidentally starting a new session on your survey.
Have you tried this? And if so, did it give you confidence in the results?
Thanks
Lisa
+19I believe I always have that option on.
How would they know that this is not a real person? A click into the survey is a click into the survey, and it gets recorded. The option they mention is useful only for anonymous surveys, I believe. It makes sure that only one response is submitted per IP address. If the links are unique per respondents, they'd have no way of identifying this unfortunately.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.