On survey data export, a large number of responses are labelled as "Survey Preview" | XM Community
Skip to main content

My survey has a problem with bot responses, but since the responses are generally:

  1. From the same location
  2. Have an abnormal duration
  3. Clustered in 1-minute intervals
  4. Have responses to short answer questions that are really short, do not show understanding of the area, or are far too academic (academic points to probably using GPT-3 or GPT-4) -- I have to manually go through these

I’ve been able to filter a lot of them using python scripts. There’s something abnormal that I realized recently. A lot of the responses are labelled “Survey Preview” in the “Status” column. I did not share the survey preview. I figured out through some basic hackery that if the following string is added to the anonymous link, it creates a preview link:
?Q_CHL=preview&Q_SurveyVersionID=current

First, this is a major oversight on Qualtrics’s part. Second, are fake responses using this trick to avoid giving IP addresses? I want to ascertain that the only way that the status would be labelled as “Survey Preview” is if they used a survey preview link.

--Kassie (she/her)

Hi, that's interesting that you are seeing a lot of responses with a status of Survey Preview. I believe the only way for that to happen is if a response is completed using the Survey Preview link, which can be easily created from an anonymous link by changing "form" to "preview" in the URL. However, Survey Previews have a Share setting that defaults to "Brand internal only" and requires login to access and I'm not sure how a bot would get around that. If this setting was changed to “Public”, make sure it is set to “Brand internal only”.

But since it seems you experiencing bots, below are 3 approaches I know about to help identify bots and prevent them from submitting a response:

1) Use a Captcha question type in Qualtrics which should prevent bots from progressing in the survey

2) Hide a question so that human respondents can't answer it but bots will, then add a branch in the survey flow to direct any responses that answered the hidden question to end of survey. A thread discussing that is here.

3) Qualtrics Fraud Detection has Bot detection and then a branch can be added in the survey flow to direct these responses to end of survey.

In checking my own surveys, that’s the exact query string for the preview link. If at some point you shared a preview link with someone instead of the anonymous link, it’s now out there gathering responses.  Can you figure out where the preview link has been erroneously shared? If you can’t, your best bet is to copy the survey, import the data from the previous survey, and make sure you only share the correct link.

Leave a Reply


Terms & Conditions