News

More questions, more insight - right? Perhaps – but survey length can be an important balancing act; too few questions and the objectives may not be met, but too many – and everything from respondent experience to data quality can be impacted. Read on for 5 key impacts of long questionnaires:

1: Respondent fatigue

Respondent fatigue describes the lessening engagement of respondents as the survey progresses, which can lead to less thoughtful answers to latter questions.

The issue can stem from a range of questionnaire design flaws, including dense question text and over-use of open-text questions, however arguably the biggest contributor is the long questionnaire.

2: Declining data quality

The impact of respondent fatigue can be seen in the data in a number of tangible ways; most notably, less variance of answers in grids (straight lining) and shorter answers to open-text questions as the survey progresses, as demonstrated by Galesic and Bosnjac (2009) in their paper “Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey”.

3: Slower fieldwork; Non-response and drop-out rates

Respondents must also be made aware of interview length prior to participating, and the longer the stated survey length, the lower the response rate.

For those who do brave the long survey, high fatigue can lead to attrition (respondents abandoning surveys part-way through) slowing fieldwork progress.

4: Bias

As well as non-response and drop-outs slowing progress, the data can also be impacted in terms of representativeness in what’s known as non-response bias.

In short, this is a kind of bias within the sample whereby potential respondents who refuse to take part, or who drop out before completion, are fundamentally ‘different’ in characteristics from those who choose to participate. Respondents deterred by a long survey may introduce this kind of bias into your data.

5: Cost

Long questionnaires can also have a financial implication, both at the design stage with research agencies, and at the fieldwork stage with sample suppliers. Respondents will need to be rewarded with higher incentives for their time, and this is typically reflected in an increased cost per interview.

How to mitigate against these impacts?

In short, fieldwork time, data quality, and respondent experience all benefit from a shorter questionnaire. However, if your survey must be longer than 15 minutes; here are some recommendations of how to mitigate the impacts:

• To reduce respondent fatigue and drop-out rates, aim to keep respondents engaged by varying question types and styles

• Reduce non-response bias by making surveys device-agnostic, and (cost-permitting), incentivising well

• And finally, minimise labour-intensive questions such as open-texts and grids, and where possible use them earlier in the survey – before respondent fatigue kicks in

Awaken your insights

Whatever your research goals -from content marketing to brand awareness, message testing to market insight - we can help you design a survey that delivers. Get in touch with us today.

Interested in receiving news like this in your inbox? Sign up for our monthly newsletter here