Large scale survey (1500 questions) | XM Community
Skip to main content

Hi!

I am relatively new to setting up large surveys and planning to build a large survey comprising many modules (approximately 1500 questions). Is there anything to note to ensure performance is not compromised given the large volume of questions?

Most of the modules will be interviewer administered. We plan to have a save and continue button and will train interviewers to have a break at the end of every module instead of in the middle.

Also, are there technical considerations that i should be aware of when building such a large survey? 

Thank you for the advice!

@bernardtancw 

Regarding the performance of the Qualtrics editor, it's advisable to divide your questions into blocks. While this doesn't impact the respondent's experience, it significantly improves the editor's usability.

Additionally, be aware that after 60+ number of items are added to a block, Qualtrics introduces a "Large Block Mode" option at the top of the block. 

Once a survey has 60+ questions and/or page breaks (including questions/page breaks in the trash), your survey will be in large block mode. This shows a reduced form of the questions. Clicking on a question will expand the question. This saves load time and minimizes scrolling distance in large surveys. At 70+ questions, all blocks in the survey will be automatically collapsed as well.

You can turn large block mode off and back on (once it has been triggered) for a block by clicking on the text next to the block name at the top of the block. When you leave the Survey tab and return to it, the setting will be enabled again.

However, in my experience, the interface tends to become cumbersome even before reaching this threshold.

Maybe you can also test a little bit with reference surveys.

Best
Christian


Hello, how are you?

I’m interested in understanding the purpose of the survey or what kind of study it corresponds to. (I don't think I've ever seen such an extensive survey.)

On the other hand, in general terms, you may want to consider some of the following points:

  1. Make every question count; don’t ask something you already know.
  2. Avoid rhetorical questions to prevent biases.
  3. Try to keep your response scales consistent (preferably from negative to positive or from smaller to larger).
  4. Be careful with absolute choice scenarios. Avoid using superlatives like "the most," "the best," and "always," as these are very subjective for the respondent and can affect how they answer the question.
  5. Use common words that are easily and consistently interpreted by your target audience.
  6. Creating blocks of questions by topic allows for smoother and quicker survey completion.
  7. Structure the survey so that each block of related questions goes from general to more specific.

Even with breaks, I believe such an extensive survey answered in a single day could lead to biases. Many people might start responding just to finish quickly.


@IRENDONJLL0 Thank you for your help!

We are doing data collection for an assessment and most of the questions are structured questionnaires which we are not able to shorten or adjust. However, with the exception of the respondent aspect, would there be any technical instability with such a survey that i should pay attention to? 

 


@chackbusch thank you for the advice. Would there be an advisable limit to the number of blocks to maintain functionality of the main survey? I have tried building a survey with 200+ questions with multiple blocks and it was working well but was a little unsure of the viability of using Qualtrics for a survey with more than 1000+ questions.


@bernardtancw Unfortunately, I can't provide any personal experience on this matter, as I have never created such an extensive questionnaire myself. I would honestly suggest addressing this question to the official Qualtrics Support. They should be able to consult with the responsible product team...


Leave a Reply