What are your pros and cons for deleting incomplete survey responses? | XM Community
Skip to main content

What are your pros and cons for deleting incomplete survey responses? 

 

For context, this is a customer experience survey, so any response is valuable. Some responses are totally blank. Some have varying completion levels, or skipped questions.

 

My thinking is…

Pros:

  1. Data is cleaner for analysis
  2. More accurate reflection of survey completion
  3. Response Quality by ExpertReview is closer to 100%

Cons:

  1. Could we lose the interaction history of the customer interaction in the XM Directory?
  2. If we decide to thank customers for their responses, will their history be lost?

I would never delete a response, partial or otherwise. You should have set a timing for when partial resposnes close out (e.g. 2 weeks after last activity). At this time, partial responses will automatically close out and blank ones will disappear. 

If someone took the time to give you any response, even to one question, that’s sentiment that you should be interested in capturing for your CX program. 

I would also point out that you may want to look at how partial vs completed sentiment is different. We noticed that partial responses are significantly less positive than completed responses on average. So if they’re not happy, they’re less likely to spend more time answering questions. 


I also agree not to delete a response, albeit I’ve heard various arguments for it once you start it’s a slippery slope on what you consider to be true responses and ends up just encouraging gamification of results, loosing true insights from customers. 

The perception of removing a customer response also doesn't sit right, consider how they would feel if they heard this.  If the decision is to remove certain responses, I’d recommend filtering them out (even displaying partial responses in another way as to not loose them) rather than deleting the record. 


I think the decision of how to handle incomplete responses really depends on the specifics of the survey and the research program. I usually set Qualtrics to delete incomplete responses rather than record. I run several different types of surveys, but the CSAT surveys I oversee are very brief, and have a very low abandon rate. My thinking is that, if they didn’t complete the survey, they either didn’t feel strongly enough about sharing their feedback to finish, or decided after starting it that their input was not applicable (a position I’ve found myself in as a respondent at times).


Thank you, @MatthewM 

I decided to keep all responses, except if the only answer was the email invitation inline question.

The thinking is that null responses could skew response rate that we report.


@Eldon another tip if you are using an inline question.  Recommend  you have bot detection turned on if you haven’t as for a little while we didn’t and learnt that some of the responses we got which only answered the inline question have been from email scanners clicking the link which registered a response rather than a human. 

We saw a significant drop in responses which only selected the inline question following this.  We deleted suspected bot responses prior to this, but now accept all responses as ‘true’ since activating the various features Qualtrics has on this.  

This is assuming you have access to these features though.  Refer to Fraud Detection (qualtrics.com) for more.

 


Hi @Eldon  - did you manage to set it up so you captured all partials apart from those just responding to the inline question, as we can’t figure out how to do that?

 

thanks 

Emma


Leave a Reply