As researchers, we are often presenting data to clients who (generally!) lap up our every word. As consumers, we are bombarded with statistics in our everyday life, anything from '8 out of 10 cats' … through to the latest economic or unemployment data. But how many of us really question what we are being told and whether or not it is giving a really accurate picture?
In a recent TED talk, Mona Chalabi asks how reliable the data we receive is, and to what extent we should question it. She takes a look behind the statistics and shares tips to help understand what the numbers are saying and identify which numbers are, and are not, reliable.
With the ongoing erosion of traditional trust structures, people are beginning to question statistics coming from purportedly (and previously) reputable sources – such as the US government. On the one hand, these are important statistics to help us understand society – but on the other, the statistics are criticised for being elitist and maybe even rigged. Mona argues that the three things to look out for are:
- How is the data communicated? Can you see the uncertainty in the numbers? Charts and sleek visualisation overstate certainty – we are less likely to question something when it is ‘packaged’ nicely, it seems more objective. So, as researchers, should we reflect imprecisions in the way we present the data – people remember the important facts but also the imprecisions
- Can you see yourself in the data? Asking where you fit in to get as much context as possible, taking the step back to look beyond the averages (which can often obscure the underlying story)
- How was the data collected? Who was asked, how were they asked, what other questions were they asked, how was the survey carried out (how unbiased was the sample). Private companies don’t have a huge interest in getting the numbers right, they just need the right numbers
Government statisticians should be totally impartial – and we need the numbers to help us make important decisions, legislate and debate issues that affect society – but we still need to check and question the numbers and look beyond the averages. An interesting talk that made me think about how I present data to my clients…