Aha, you’re thinking, I know this. It’s when it’s sugging (selling under the guise of a survey).
Consider, however, the following scenario. A person is invited to take part in an online survey. They agree, perhaps because they are keen to share their opinions and experiences. Or maybe they are a naturally co-operative soul who likes to be helpful. Whatever the motive, their interest is captured and they answer a series of questions until, out of the blue, they are told they are ‘not eligible’.
So now you’re thinking, well, obviously, the researcher was looking for certain types of people or had already filled parts of the quota. Obvious to you because you are a researcher and you know all about sampling and screening. But how is that process seen by survey participants? Having agreed to help or have their say, how do they feel about being judged not fit for inclusion?
I ask because of some of my own recent experiences as an online research participant, when I’ve been ejected from surveys as ‘not eligible’. Not because of an industry screener that barred market researchers, but for reasons the survey designers didn’t bother to explain.
One online survey, in particular, made me very cross. It was for a well-known retailer. I’d volunteered to take part in further research when I completed a pop up survey on the retailer’s website. When an invitation arrived by email, I clicked on the link and got stuck in.
Several screens of questions asked me how often I shopped at the retailer’s various departments, how recently I’d bought certain products and how I felt about the retailer, my likelihood of recommending them to friends and family. I didn’t keep count of the questions but there were lots of them, screen after screen. Then WHAM! I suddenly wasn’t eligible to take part.
And I thought, but surely I already did take part?
This particular set of questions might be an extreme example, surveying the customer experience in some detail under the guise of screening. Sugsing, perhaps? It was especially a concern because participation was incentivised by a prize draw. MRS regulations state that such incentives relate only to agreeing to participate, they must not be conditional on completing a whole interview. This retailer’s survey didn’t say whether ineligibles were going to be entered into the prize draw but, since no contact details were taken, it seemed that MRS rules may have been breached. Reason enough to make a researcher frown, but likely to make other participants think ‘Swiz’.
My point, though, is that lengthy screeners in online surveys are not uncommon and the process of closing when participants are out of scope is often rather abrupt. The very phrase ‘not eligible’ feels like a slamming door.
For interviewer administered interviews we write introductory scripts that explain if we are surveying people with certain characteristics (‘I just need to ask a few questions to check…’) and let people down gently if they don’t meet the requirement (‘I’m sorry, we’re looking for people with/we’ve already interviewed lots of people with…’). A phone survey might even disguise rejection: ‘That’s all I need to ask, thank you very much for your help’. But, for some reason, we seem inclined to forget our manners when designing online survey scripts.
We believe that participation rates are higher in face-to-face surveys than in other methods, because it’s harder to say ‘no’ to someone standing in front of you. Just as it’s easier to reject a faceless researcher, maybe it’s also easy, when designing online questionnaires, to forget that our invisible participant is a real live person, who deserves to be treated with courtesy.
If we want to maintain public goodwill towards market research, researchers and their clients need to look at the experience through the eyes of the people invited to take part. If an online survey has screening or quota control questions, we should explain this, as we would in an interviewer administered survey, and keep the screening short: don’t waste participants’ time asking questions that don’t have a direct bearing on eligibility.
Remember that an online questionnaire is not just a chunk of computer code, it’s an interview with a person. A person who doesn’t know about the distinction we have drawn between screener and main interview, but who believes they are taking part in the survey from the moment they read the first question. So if it’s necessary to close the interview early, do it politely: it must surely be possible to come up with something less dismissive than ‘not eligible’.
The best way to see the survey research experience through participants’ eyes is for researchers and clients to be participants themselves, at every opportunity (industry screeners permitting). And then see how YOU like it!
A copy of the MRS Regulations for Administering Incentives and Free Prize Draws is available by clicking on the link on the right.