Which audience would be appropriate for calculating our response rate? And what is an adequate sample size? | XM Community
Skip to main content

I work for an event company. We send email surveys to guests after they attended to collect feedback on their experience at the event. The email survey is only sent to 1 person per order, not every person on that order, as we only get the email address for the person who submitted the order. Does this mean I should only be drawing conclusions about the Tickets Purchasers and not all Event Attendees, since not all Event Attendees get the email? 

Also, can anyone offer any guidance as to what a good response rate/sample size is? For all Event Attendees, the rate is 5%, for Ticket Purchasers alone it is 15%. 

Thanks in advance for your insights!

@ChelseaIC there is no perfect science for this one, but as you rightfully call out the experience from those who purchased vs those who attended (with someone else purchasing) may have different experience.  So you should be cautious to conclude the experience is the same.  You would need to gather feedback from attendees and compare with ticket purchaser to see if the experience varies.

The right sample size, is typically a question of if the responses are statistically significant in representing the base.  The % who need to respond varies depending on the overall size of the base they represent.  Margin of Error is what you use to calculate this & to understand what is within your tolerance.  Refer to Your Guide to Margin of Error (With Calculator) - Qualtrics

5 & 15% response rate also isn’t a bad response rate, but you will see it’s a common question raised, very subjective, and no easy way to benchmark.  Our programs vary between 3 to 30% response rates as an example.  

If the volume of event attendees is quite small to begin with, you may also find you may not be able to get a statistically significant volume, as this is quite hard when you start with a small base.

@ScottG Thanks, Scott, for your response! Moving forward, I will be sure to communicate to my team that the survey results are reflective of the Ticket Purchaser experience, not all Event Attendees. And the link you provided was very helpful for figuring out our sample size. Thank you!

@ChelseaIC glad it assisted. 

PS. Worth keeping in mind experience could also be similar, so don’t totally disregard it if you feel it might represent both groups.

Personally, I am always happy to report data and themes even if it isn’t of statistical significance.  Just flag when it’s based on low sample and isn’t statistically significant, so data is viewed as an indication only and with caution.  Call out any unknowns or hypothesis with it.  Too much insight would be missed if we always waited for statistical significance across most of our programs, however it helps to manage expectations and addresses anyone who is statistically minded.

@ScottG Those are great notes to keep in mind - Thanks for the additional thoughts!

Leave a Reply


Terms & Conditions