Latest research and updates
Five impacts of a long survey
10 Aug 2021
More questions, more insight - right? Perhaps – but survey length can be an important balancing act; too few questions and the objectives may not be met, but too many – and everything from respondent experience to data quality can be impacted. Read on for 5 key impacts of long questionnaires:
1: Respondent fatigue
Respondent fatigue describes the lessening engagement of respondents as the survey progresses, which can lead to less thoughtful answers to latter questions.
The issue can stem from a range of questionnaire design flaws, including dense question text and over-use of open-text questions, however arguably the biggest contributor is the long questionnaire.
2: Declining data quality
The impact of respondent fatigue can be seen in the data in a number of tangible ways; most notably, less variance of answers in grids (straight lining) and shorter answers to open-text questions as the survey progresses, as demonstrated by Galesic and Bosnjac (2009) in their paper “Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey”.
3: Slower fieldwork; Non-response and drop-out rates
Respondents must also be made aware of interview length prior to participating, and the longer the stated survey length, the lower the response rate.
For those who do brave the long survey, high fatigue can lead to attrition (respondents abandoning surveys part-way through) slowing fieldwork progress.
As well as non-response and drop-outs slowing progress, the data can also be impacted in terms of representativeness in what’s known as non-response bias.
In short, this is a kind of bias within the sample whereby potential respondents who refuse to take part, or who drop out before completion, are fundamentally ‘different’ in characteristics from those who choose to participate. Respondents deterred by a long survey may introduce this kind of bias into your data.
Long questionnaires can also have a financial implication, both at the design stage with research agencies, and at the fieldwork stage with sample suppliers. Respondents will need to be rewarded with higher incentives for their time, and this is typically reflected in an increased cost per interview.
How to mitigate against these impacts?
In short, fieldwork time, data quality, and respondent experience all benefit from a shorter questionnaire. However, if your survey must be longer than 15 minutes; here are some recommendations of how to mitigate the impacts:
- To reduce respondent fatigue and drop-out rates, aim to keep respondents engaged by varying question types and styles
- Reduce non-response bias by making surveys device-agnostic, and (cost-permitting), incentivising well
- And finally, minimise labour-intensive questions such as open-texts and grids, and where possible use them earlier in the survey – before respondent fatigue kicks in
Awaken your insights
Whatever your research goals -from content marketing to brand awareness, message testing to market insight - we can help you design a survey that delivers. Get in touch with us today.
Interested in receiving news like this in your inbox? Sign up for our monthly newsletter here
In IT Together blog: We three researchers of Vanson Bourne are...
04 Dec 2023
Discover the latest insights in our monthly blog series. This month: a look back at some of 2023's tech trends
In IT Together blog: The devil is in the data
31 Oct 2023
Discover the latest insights in our monthly blog series. This month: an ITDM insider perspective on data security and risks
In IT Together blog: Research-as-a-recipe for marketing planning
29 Sep 2023
Discover the latest insights in our monthly blog series. This month: how research insights on customer challenges can enhance your...
In IT Together blog: Measuring perceptions of tech leaders
24 Aug 2023
Discover the latest insights in our monthly blog series. This month: how industry tech leaders are perceived
In IT Together blog: Will ChatGPT be a friend or foe to IT decision makers?
21 Jul 2023
Discover the latest insights in our monthly blog series. This month: ITDM views on Chat-GPT