By Deborah Sleep, director, Engage Research and Jon Puleston, vice president, GMI
Market researchers conduct more and more of their research online, but if questionnaires don’t engage with respondents, data quality can be seriously compromised.
Engage Research, a full-service consumer market research agency, in collaboration with GMI, a leading provider of global panels and technology-enabled services to the global market research industry, interviewed 3,600 online panellists, analysed the results of some 550 surveys, and conducted a number of research experiments to first, understand what happens when respondents get bored and second, identify some solutions to these challenges.
Bored respondents can affect research data in various ways. We created a 15-minute test survey, where we moved questions around to see how they were answered at the end of the interview compared to the beginning. We recorded a series of important results, as follows:
· Speeding – 22% less time spent reading instructions and 17% less time answering questions
· Pattern answering – 38% increase in people answering in a mechanical way
· Loss of granularity - 18% increase in neutral responses and 25% drop in use of the extremes of a five-point scale
· Verbatim loss – 41% drop in the number of words written
These issues can have a serious impact not just on data quality, but on the willingness of respondents to complete surveys at all.
So, to help minimise the impact of boredom on survey data, we’ve identified 10 golden rules to help address these key issues:
1. Keep it short
Twenty minutes is a good rule of thumb for maximum survey length. Boredom and drop-out levels increase greatly beyond this point.
2. Make a positive first impression
The first five minutes of the interview are critical. Beyond that threshold, respondents are far more likely to complete the whole interview. Keep screening and profiling questions to a minimum at the start of the survey, and expose respondents to the real subject-matter early on.
3. Don’t repeat yourself
Repetitive questions encourage formulaic response patterns. Instead, consider asking a larger sample to complete a shorter exercise, rotating stimulus through the sample or mixing question styles up.
4. Prune lists
We noted a huge increase in drop-out for questions showing more than 20 response options on a page. Challenge each response option, examine ways of dividing the list, and use visuals to make your survey more engaging.
5. Avoid drop-down menus
Think back to the last time you had to fill in your country of residence in an online form. Did you, like me, scroll down to the ‘U’s looking for UK, then up to the ‘G’s for Great Britain, up again to ‘E’ for England, etc.? When asked, 75% of online respondents said they disliked this form of question.
6. Use grids with care
Whilst they can generate great data with powerful analytical possibilities, grids cause 80% more drop-out than other question formats, and respondents rated them only 3 ½ out of 10. Alternative formats we piloted in our research, using animated alternatives or drag-and-drop mechanisms, were preferred by respondents (rated 8+ out of 10) and generated much more granular data (less pattern answering and fewer neutral responses).
7. Take care with tick boxes
Tick box questions generate 36% more drop-outs than other question formats, and are subject to both order and speeding effects. In our tests, simply splitting a list of 30 items over two screens increased the click count by 18%, and using drag-and-drop mechanics combined with visuals increased responses by 45%.
8. Engage and entertain
Using animated introductions or stimulus can encourage respondents to spend more time on a survey. It should be used with care as it can affect the way people respond, but in our experiments, we used it to encourage panellists to spend twice as long answering, generating almost twice as much response. Changing question wording to suggest more of a challenge, or using a timer to turn a question into a game, increased word counts by as much as 300%.
9. Use technology to enhance the respondent experience
Simply moving a basic questionnaire from an HTML to a Flash® programming platform yielded significantly better respondent feedback. The interview was easier to complete, the questions came across as more interesting, and respondents happily spent longer answering them.
10. And we would say this, but …
The answers you get can only ever be as good as your sample, your questions and the quality of your analysis. Use professional research agencies and quality panels to ensure that you get what you need and generate quality research results you can use for your strategic business decisions.
Check out 12ahead, our brand new platform
covering the latest in cutting-edge digital marketing and creative technology from around the globe.
12ahead identifies emerging trends and helps
you to understand how they can apply to modern-day companies.
We believe 12ahead can put you and your
business 12 months ahead of the competition. Sign up for a free trial today.