When conducting market research, there can be a tendency to take everything that comes out of an interviewee’s mouth as gospel. Yet it’s not that black and white.
One is not dealing with facts such as 2+2=4, but opinions, future predictions, perceptions and sketchy recollections. So what respondents say can sometimes be unreliable, and it’s our job as insight specialists to try and minimise this.
Below are just a few of the key causes of the problem and ways to help mitigate them:
Sketchy memories
I was recently tasked with conducting research on some products from last summer's range, in readiness for sprucing up this year's offering. I asked consumers about their decision process, what they finally chose and why. Although they had a broad notion of what they probably did based on their personality and preferences, they found it extremely hard to recollect exactly what they were thinking and doing some six months ago with any accuracy. And while able to comment on their general buying behaviour, they may well have forgotten impulse actions on the day such as ‘I was just in the mood for it’ … ‘I don’t normally buy it but I spotted a great offer’.
I didn't have the luxury of picking the timing on the above occasion, but to get nearer to the truth it’s important, where practical, to time the study as close to the purchase/event as possible. This might involve undertaking exit interviews as people leave a shop, or customer satisfaction interviews straight after a call to a contact centre. Otherwise, it’s surprising how much and how quickly people forget. This is especially true if it’s a fairly inconsequential event. You may remember everything about the excitement of getting your new car, but not the detail of buying a bar of soap or calling to give your meter reading!
Unreliable predictions
I’m often asked to research consumer reaction to a new product, if they’d buy it and how much they'd be willing to pay. Again, this can be an area fraught with the potential for unreliable feedback unless careful. One of the reasons is that chatting in a focus group is very different to actually putting your hand in your pocket. It can be easy to hypothesize about what you might do, but at that point you’re not standing in the shop with a limited budget, competing priorities or other brands to tempt you.
There are several ways to overcome this, largely based on ensuring that answers are put into context and grounded in reality (accepting that there are of course times when we don’t want to constrain people’s imaginations). For example, if conducting an in-store interview, instead of asking shoppers to select chocolates they ‘would like to buy’, one might ask them to choose based on the budget they usually have, the time they normally take or the occasion they'd be buying for, eg: weekend treat, etc. Providing a frame of reference means that how they imagine they’d behave is likely to be closer to reality, rather than them thinking: ‘This is a fun exercise, I think I’d have one of those, one of those and one of those!’
The danger of post-rationalising
Experts say that as much as 90-95% of our decisions are made subconsciously. Even if buying a simple tin of custard, we may think we can easily articulate why we bought it, eg: ‘I saw a good offer’. Yet a busy shopper will often be making such choices in a nano-second, and their glib post-rationalised explanations often mask a host of other subtle subconscious factors at play. So, dig a little deeper into the quick custard decision and it could well transpire that ‘Ambrosia was the brand my mum always bought … it’s a comfort food that reminds me of home … the graphics make it look like a quality product … my husband gets cross if I go cheap and buy own label’, etc.
There are various techniques for getting under the skin of what’s really going on in consumers’ minds, too lengthy to go into here. But a basic principle is not to take their first, top of mind answer at face value. Aim to scratch the surface and ask ‘Why is that so important? What difference does that make to you? What else matters? Could other brands also do this?’ Are there any you wouldn’t choose, no matter how cheap? Why not’ etc.
All in all there can be many reasons why people don’t always say what they mean, or mean what they say, beyond those above. Other factors include …
The ‘research effect’ – eg: analysing a marketing leaflet for an hour in a focus group, when in real life a consumer would ever accord it such scrutiny
Peer group pressure – a desire to please or say the right thing, when this may not happen in reality eg: 'Yes, like you, I give my child a healthy breakfast (I daren’t admit I give them Cocopops!)'
The list of considerations goes on. In all such cases it’s our role as research experts – in both designing the research and analysing the findings – to factor in these pitfalls and find ways to minimise unreliable testimony, ultimately helping our clients get much closer to the truth.