Latest research and updates
Five impacts of a long survey
10 Aug 2021
More questions, more insight - right? Perhaps – but survey length can be an important balancing act; too few questions and the objectives may not be met, but too many – and everything from respondent experience to data quality can be impacted. Read on for 5 key impacts of long questionnaires:
1: Respondent fatigue
Respondent fatigue describes the lessening engagement of respondents as the survey progresses, which can lead to less thoughtful answers to latter questions.
The issue can stem from a range of questionnaire design flaws, including dense question text and over-use of open-text questions, however arguably the biggest contributor is the long questionnaire.
2: Declining data quality
The impact of respondent fatigue can be seen in the data in a number of tangible ways; most notably, less variance of answers in grids (straight lining) and shorter answers to open-text questions as the survey progresses, as demonstrated by Galesic and Bosnjac (2009) in their paper “Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey”.
3: Slower fieldwork; Non-response and drop-out rates
Respondents must also be made aware of interview length prior to participating, and the longer the stated survey length, the lower the response rate.
For those who do brave the long survey, high fatigue can lead to attrition (respondents abandoning surveys part-way through) slowing fieldwork progress.
As well as non-response and drop-outs slowing progress, the data can also be impacted in terms of representativeness in what’s known as non-response bias.
In short, this is a kind of bias within the sample whereby potential respondents who refuse to take part, or who drop out before completion, are fundamentally ‘different’ in characteristics from those who choose to participate. Respondents deterred by a long survey may introduce this kind of bias into your data.
Long questionnaires can also have a financial implication, both at the design stage with research agencies, and at the fieldwork stage with sample suppliers. Respondents will need to be rewarded with higher incentives for their time, and this is typically reflected in an increased cost per interview.
How to mitigate against these impacts?
In short, fieldwork time, data quality, and respondent experience all benefit from a shorter questionnaire. However, if your survey must be longer than 15 minutes; here are some recommendations of how to mitigate the impacts:
- To reduce respondent fatigue and drop-out rates, aim to keep respondents engaged by varying question types and styles
- Reduce non-response bias by making surveys device-agnostic, and (cost-permitting), incentivising well
- And finally, minimise labour-intensive questions such as open-texts and grids, and where possible use them earlier in the survey – before respondent fatigue kicks in
Awaken your insights
Whatever your research goals -from content marketing to brand awareness, message testing to market insight - we can help you design a survey that delivers. Get in touch with us today.
Interested in receiving news like this in your inbox? Sign up for our monthly newsletter here
In IT Together blog: The importance of speaking the same language in the B2B tech space
27 Aug 2021
Discover the latest insights in our monthly blog series. This month: the importance of speaking the same language in the...
In IT Together blog: Out of the office but still online
21 Jul 2021
Discover the latest insights in our monthly blog series. This month: the struggle to switch off from work and technology....
In IT Together blog: The human side of technology investment
29 Jun 2021
Discover the latest insights in our monthly blog series. This month: the human side of technology investment.
In IT Together blog: Working for wellbeing - an ITDM perspective
21 May 2021
Discover the latest insights in our monthly blog series. This month: wellbeing in the workplace.
In IT Together blog: Total Experience - new kid on the block or transient imposter?
29 Apr 2021
Discover the latest insights in our monthly blog series. This month: the concept of the Total Experience.