Patrick Fagan is a consumer psychologist with nine years' experience helping brands 'turn mind into money' – that is, practically applying psychological science. He lectures at Goldsmiths, University of London and University of Arts London (UAL).
He has taken a look at the impact that professional participants have on online panels – and his conclusions do not make happy reading. Not only is the sample skewed, the quality of the actual responses is also compromised. Whilst some of the issues with sampling can be mitigated through careful weighting, the quality issue is harder to both measure and address.
He outlines three main ways in which the research may be compromised:
- Habitual responding which results in low effort heuristic thinking
- Implicit memory – habitual respondents learn patterns in survey design which may impact on how they respond to certain questions
- Motivation – somewhat counter-intuitively, people who are being paid to take the survey are less likely to put in as much cognitive effort than those who have some other interest in the survey
His article raises significant questions for the provision and use of data derived from online panels for all researchers – and with the shift towards panels, the problems will only increase in the future.
Bearing in mind that the article is sponsored by LiveMinds, the article concludes that behavioural recruitment is the way to go. It is nonetheless interesting reading and does identify some real issues. Read his full blog for more.