Honest responses – managing client’s expectations

Scoping the issue:  An Insight Manager at a big FMCG company has asked for qualitative research that gives more 'honest responses' when developiing brand communications ideas.  In early-stage developmental research of Brand Campaign Ideas and Executions (usually rough TV ads, activation ideas, etc.), consumers often provide ostensibly positive responses – they like the work, they seem to get the intended communication, they think it is distinctive and credible, and so on. 

However, when the work finally gets made, real world responses – as measured in quantitative post-testing and indirectly via brand tracking, sales performance, etc – suggest that the work hasn’t turned out to be that good after all.  What can be done to give the marketing team more confidence that they will get the ‘honest feedback’ they crave?

The e-group brain's response broadly fell into 3 areas:-

1.  The client misdiagnosing the issue, in particular failing to take into account the huge changes that happen to ideas between qual, quant pre-testing and final execution

This certainly can be an issue with clients and often one which they are oblivious to, or would rather not own up to.  It’s much easier to blame the researcher than examine their own shortcomings.  Key inputs are:-

• Resist the temptation to assume that this is due to participant honesty or the qual research process and get the client to consider the bigger picture.  Particularly the massive changes that are inflicted on material between concept/ rough execution and finished execution
• Early stage development ideas are inevitably different to the 'thing' once it is developed and no known research can ever predict how the final 'thing' will fare (the inconvenient truth about research!)
• The creative interpretation and treatment of the 'great' idea or concept is often at odds with how consumers imagine it
• The primary reason for the discrepancy is the difference between the finished ad that respondents envisage compared to what is finally produced. Having done numerous animatic and finished film quant pre-tests, you can also get similar differences there
• The nugget of an exciting idea can be lost in later stages through poor translation of the true emotional essence and the idea that consumers warmed too.  The skill of the researcher is to ensure that the nugget is clearly described by what it is and what it is not ie. we can help the client by defining the parameters
• The advertising that eventually gets made very often bears little relationship to what one researched in qual anyway, since few agencies or clients seem to have anyone responsible for ensuring that the features that made the idea work are preserved in the finished article
• In the time between ideas examined in qual and ideas brought to market the market itself may have moved on. FMCG is renowned for this – the context in which consumers make their purchases is littered with competitor new product launches, price changes, price promotions, variations in product availability, etc etc
• Context is all, with the context of qual exploration being very different to the context of the quant post-testing and brand tracking.  It is likely that the quant work takes a whole market approach, comparing brands; and a whole marketing mix approach, looking at the ads in conjunction with the product, the price, preferences against competitors, etc.

2.  Encouraging the client to better understand the what qual is, the role of consumer feedback in the process, and how qual should be used

For many clients, this is potentially the nub of the issue.  Qual is viewed as a 'predictive test' of how ideas will fare in the 'real world'.   As such, consumer responses are taken at face value and the importance of the analysis and interpretation is not registered.  Some of the key inputs were:-

• In early stage creative development qual, you can only really talk about potential and relevance. The clue's in the name: 'creative development research'.  The idea is that the best of the ideas get through and are steered away from the bigger rocks visible at that stage.  But getting the execution right is a whole different ballgame
• If actionable insight and clear meaning are the finished dish of a qual process, viewing groups is watching the raw ingredients being prepared
• If the marketing team can't manage to understand not to take consumer response at face value, are allowed to rush off and act on what they've seen in the groups and won't take on board guidance on how to respond to groups, then the client has bigger potential issues
• One hypothesis is that much of the issue lies in the increasing tendency of businesses to recruit graduates from Marketing or Business courses, rather than those from the classic humanities or even science disciplines, meaning that marketing (and even advertising) is increasingly populated by people who are very hot on process but very weak on the ability to think

3.  Tools and Techniques for getting to more 'honest' consumer responses

These include some reminders of good practice…

• Positioning qualitative research and its limitations/ uses corretly.  Consumer feedback in qual should never be taken at face value and that there are, of course many reasons why respondents react more positively to ideas in research than in the 'real world' including:-
  – they spend longer considering and 'getting into' the ideas
  – they have fewer distractions, focusing on the ideas and nothing else
  – they want to 'please' the moderator/ client
• – they succumb to group effect, meaning for example that participants play off each other and look like they’re enjoying the ideas, when in fact they are enjoying the sense of belonging, socialising role play, forming relationships, etc
• Not underestimating the importance of the warm-up part: it may also seem a luxury to spend time getting to know people and doing proper introductions, giving reassurances about confidentiality, saying it doesn't matter if they hate the ideas as you don't work for the company, a few light jokes to put the room at ease etc.
• The set up to the research needs to ensure that the respondents know that they are speaking to a researcher who has no stake in the product ( so they don't feel they are hurting anyone's feelings).  When the researcher feels that the respondents mean no when they say yes, they should challenge the respondents to be honest – this is about emphasising our experience as researcher and need for proper analysis and reporting
• One of the core skills of any qualitative researcher is to be able to spot these things and to take account of them when analysing and interpreting feedback – and we need to be given sufficient time to do this properly
• Managing and educating marketing managers better so that they do not jump to conclusions or take responses at face value – a herculaen task reminiscent of the painting of the Forth bridge

…as well as some more novel approaches:-

• •trying to put participants in as real world a context as possible without letting them know the focus of the research would be a good / interesting exercise
• Eye tracking
• Facial coding
• ECG
• Psychometric techniques
• Neurometric techniques (EEG, FMRI)
• Biometric techniques (Eyetracking, Facial coding, Galvanic Skin Response)
• Implicit Association Tests