Latest research and updates
Five impacts of a long survey
10 Aug 2021
More questions, more insight - right? Perhaps – but survey length can be an important balancing act; too few questions and the objectives may not be met, but too many – and everything from respondent experience to data quality can be impacted. Read on for 5 key impacts of long questionnaires:
1: Respondent fatigue
Respondent fatigue describes the lessening engagement of respondents as the survey progresses, which can lead to less thoughtful answers to latter questions.
The issue can stem from a range of questionnaire design flaws, including dense question text and over-use of open-text questions, however arguably the biggest contributor is the long questionnaire.
2: Declining data quality
The impact of respondent fatigue can be seen in the data in a number of tangible ways; most notably, less variance of answers in grids (straight lining) and shorter answers to open-text questions as the survey progresses, as demonstrated by Galesic and Bosnjac (2009) in their paper “Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey”.
3: Slower fieldwork; Non-response and drop-out rates
Respondents must also be made aware of interview length prior to participating, and the longer the stated survey length, the lower the response rate.
For those who do brave the long survey, high fatigue can lead to attrition (respondents abandoning surveys part-way through) slowing fieldwork progress.
As well as non-response and drop-outs slowing progress, the data can also be impacted in terms of representativeness in what’s known as non-response bias.
In short, this is a kind of bias within the sample whereby potential respondents who refuse to take part, or who drop out before completion, are fundamentally ‘different’ in characteristics from those who choose to participate. Respondents deterred by a long survey may introduce this kind of bias into your data.
Long questionnaires can also have a financial implication, both at the design stage with research agencies, and at the fieldwork stage with sample suppliers. Respondents will need to be rewarded with higher incentives for their time, and this is typically reflected in an increased cost per interview.
How to mitigate against these impacts?
In short, fieldwork time, data quality, and respondent experience all benefit from a shorter questionnaire. However, if your survey must be longer than 15 minutes; here are some recommendations of how to mitigate the impacts:
- To reduce respondent fatigue and drop-out rates, aim to keep respondents engaged by varying question types and styles
- Reduce non-response bias by making surveys device-agnostic, and (cost-permitting), incentivising well
- And finally, minimise labour-intensive questions such as open-texts and grids, and where possible use them earlier in the survey – before respondent fatigue kicks in
Awaken your insights
Whatever your research goals -from content marketing to brand awareness, message testing to market insight - we can help you design a survey that delivers. Get in touch with us today.
Interested in receiving news like this in your inbox? Sign up for our monthly newsletter here
In IT Together blog: A lightbulb moment
08 Jun 2022
Discover the latest insights in our monthly blog series. This month: how sustainability is shaping strategy.
In IT Together blog: World Password Day
05 May 2022
Discover the latest insights in our monthly blog series. This month: the pitfalls of passwords.
In IT Together blog: Good enough isn't good enough
28 Apr 2022
Discover the latest insights in our monthly blog series. This month: the impact of poor people management issues in the...
In IT Together blog: OpportunITy awaits
31 Mar 2022
Discover the latest insights in our monthly blog series. This month: how the IT jobs market is changing.
In IT Together blog: Who trusts tech?
25 Feb 2022
Discover the latest insights in our monthly blog series. This month: how trust matters in the tech industry.