The ICG’s Tom Woodnutt (this year’s ICG Independents Award winner) recently commissioned a review of psychology literature to see what he could pick up on both online and face to face qualitative approaches. He compares and contrasts the approaches and shares some of his findings here.
Online vs face-to-face qual
Any qualitative researcher who tells you that face-to-face is categorically better than online, is wrong (and vice-versa)! Every choice between methodologies involves a trade off. Each has its own advantages and disadvantages. That said, I find asynchronous online and mobile qual indispensable to the type of customer experience and concept development work that my consultancy Feeling Mutual often does.
As a specialist in online and mobile qual (and a psychology graduate) I wondered what academia could teach us about how to do online qual better. So I commissioned a review of the psychology literature on the subject of online qualitative research and related topics (such as cognition, memory and emotional disclosure). In this article, I share some of what I found and also what I’ve experienced in 20 years as a qualitative researcher (14 of which have included online).
For reference, by ‘online qual’ I mean both webcam group discussions in real time and asynchronous text based feedback that runs over a few days.
Learn more about the psychology of online qual
If you’d like to learn how to apply insights from psychology to do better online qualitative research you can take part in my training on the subject which is here.
The advantages of online qual over face to face groups (according to academia)
a. People are less inhibited and more emotionally open in online qual
Several academic studies suggest that people are liberated by the remoteness of online qual (particularly via asynchronous text because people feel relatively anonymous) and so they will make more emotional disclosures. 1,2,3 They feel more at ease online compared to the daunting context of an in-person discussion in a formal viewing facility4 sat next to a bunch of strangers and a two-way mirror!
Similarly, research has shown how communication in dating apps is associated with people making more emotional disclosures in their texts compared to in their face to face meeting on a first date5). Taking part in qual online means participants are in a familiar, safe, natural, comfortable environment which makes them feel less self-conscious or intimidated and more open6.
b. There are fewer ‘researcher effects’ online
In qualitative research the moderator can sometimes (unintentionally) influence what participants say. For example, participants might shape their responses based on what they think the moderator wants to hear, or if they have been rubbed up the wrong (or right) way by them. Studies have suggested that because online qual is remote and there is a greater distance between the moderator and participants, the risk of moderator influence is lower7,8. So this means participants are more likely to share what they individually think and feel.
c. There are less ‘group effects’ online
Participants are also less likely to influence one another in asynchronous, text based online qual, compared to in face-to-face group discussions. This is because you can set it so that participants’ answers remain private to one another. Whereas in real-time group discussions, studies have shown that if the group sees a participant as ‘higher status’, then those who feel lower status might simply default to their opinion. Or conversely they might disagree with them for the sake of asserting themselves, rather than because they necessarily believe in what they’re saying9.
A phenomenon called ‘collaborative inhibition’ asserts that ‘lower status’ people in a group discussion will sometimes get distracted from their own memory as other ‘higher status’ people share theirs10. So in effect they forget what they think because they assume it’s not valuable (once the higher status person has said theirs). This effect is also removed by the privacy settings in asynchronous online qual.
d. More time for more considered questions and answers
In asynchronous online qual there is more time for the moderator and participants to develop their thoughts, introspect and make more considered comments, because the conversation is not in realtime and so there’s less time pressure. This can lead to more thoughtful contributions by both moderators and participants3. And it means the moderator can develop more sophisticated interpretations.
The advantages of online qual over face to face groups (according to experience)
Other advantages of (mostly asynchronous rather than webcam) online qual over face to face groups that I have experienced (but which I couldn’t find reference to in the literature) include:
e. More depth (since you get more feedback per person, since they do not have to wait for others to stop speaking before they can)
f. More vivid, contextual outputs (though mobile video and photos taken from natural environments)
g. Lower stress (by not having to give up your evenings to run focus groups or deal with the pressure of being viewed by clients)!
h. More agile (as insights are instantly available in text based feedback).
Of course, there’s always another side to every story! Here’s what the psychology literature (and my experience) says about the advantages in face to face qualitative research over online.
The disadvantages of online qual over face to face groups (according to academia)
a. Greater ’cognitive load’ online
In webcam groups, people can get distracted by the cognitive challenges inherent in communicating through the medium. These include having to work out when it’s your turn to speak and what pace is appropriate to speak at (as well as managing the tech)11. This cognitive load can result in less considered and more automatic responses.
The ‘Heuristic Systematic Model’ (HSM) theory explains this. People are more likely to take a mental shortcut when answering a question if they are under cognitive strain. So in webcam discussions (which uses up more of participants’ mental capacity), people are more likely to be influenced by superficial details such as the moderator’s appearance or likeability than by the quality of the information they are sharing (compared to in face-to-face interaction)12. Here’s some more information on the study from another article I wrote on the subject. So this means that webcam groups can lead to more superficial responses than face to face groups.
b. Less body language is available online
In online qual there are fewer non-verbal cues available (such as body language, micro-expressions and that ethereal feeling in the room). So this reduces a layer of communication (and efficiency) which can result in a less nuanced read on what people are really feeling and trying to say13.
c. Weaker moderator control and rapport online
It is also harder to build rapport with participants online14. Rapport is essential to a constructive relationship in a group discussion because it motivates people to focus, engage, be honest and helpful. This can reduce the power of the moderator to inspire a constructive discussion and increase participant motivation15.
d. Higher risk of drop out
The risk that participants might drop out (due to choice or technology failure) or not participate fully (due to distraction and lack of rapport16), is higher online as there is a lower barrier to exit2.
The disadvantages of online qual over face to face (according to experience)
Although I couldn’t find a reference to these in the literature, from experience there are other shortcomings in online qual compared to face to face (mostly relating to webcam rather than asynchronous online qual).
a. Weaker client-communication
An unofficial benefit of face-to-face group discussions is the opportunity to talk with the client and agency in the backroom of the viewing facility during the sessions. This channel helps you understand the nuances (often political) of the client’s challenge. There is less opportunity for this information to be shared in online qual (unless you actively create the space for it). This reduces the chance for client, agency and researcher relationships to develop and for the research to get closer to their needs and perspectives.
b. Less time with local moderators
In face-to-face research, when you travel to another market for international fieldwork, you will get more face time with the local moderator, compared to in online qual. That means there is less opportunity to learn about local nuances from the local moderator’s perspective, in online qual.
c. Less memorable moderation experience
It is easier to remember what people say when you experience it in person. Personally, I believe this is because you process what they say on a deeper emotional level when in person, compared to when you are watching them say it on a screen or reading it in a text response. I think this makes analysis more intensive in online qual because you don’t have as strong an instinct for what matters when you don’t hear it in the flesh.
d. Less spontaneous interaction online
Also I have found participants are less likely to bounce ideas off each other online. This can be useful if you want them to develop ideas together or see how they interact with one another.
In conclusion…
Qualitative researchers will always have their preferred way of running group discussions – and we should pick the right tool for the task. For some it will be online, for others it will be in person.
Personally, for the projects I tend to do (which often require getting feedback from people in their natural context, and getting their reactions to concepts without being influenced by others) on balance, I prefer asynchronous text and mobile based online qual over face-to-face and webcam qual. That’s because asynchronous gives me more feedback from in the moment (which is more vivid and less prone to recall bias), it is more valid than group discussions (because people are not influenced by what others say as they don’t see or hear them), it is more in depth (as you get more feedback per person than in a group discussion) and it is better for agile client / agency workflows (because you have instant transcripts and can open up analysis more easily).
If you think that this account is missing any other important pros and cons of on vs offline qual, please add them in a comment and share your perspective!
If you want to find out how to overcome the above shortcomings in online qual (both webcam and asynchronous) then please check out my training about how ‘Psychology can inspire better online qual’ which is here.
1. Joinson, A. (2005). Internet behaviour and the design of virtual methods. In C. Hine (Ed.), Virtual methods: Issues in social research on the Internet (Vol. 21–34, pp. 21–34). Oxford: Berg.
2. Tates, K., Zwaanswijk, M., Otten, R., van Dulmen, S., Hoogerbrugge, P.M., Kamps, W.A., Bensing, J.M., 2009. Online focus groups as a tool to collect data in hard-to-include populations: examples from pediatric oncology. BMC Med Res Methodol 9, 15.
3. Fox, F.E., Morris, M., Rumsey, N., 2007. Doing Synchronous Online Focus Groups With Young People: Methodological Reflections. Qual Health Res 17, 539–547.
4. Fielding, N. G., & Fielding, J. (2011). User satisfaction and user experiences with Access Grid as a medium for social science research. International Journal of Social Research Methodology, 15, 355–367.
5. Baker, A.J. (2005). Double click: romance and commitment among online couples. Cresskill, NJ: Hampton Press.
6. Weller, S., 2017. Using internet video calls in qualitative (longitudinal) interviews: some implications for rapport. International Journal of Social Research Methodology 20, 613–625
7. Mann, C., & Stewart, F. (2000). Internet communication and qualitative research. Thousand Oaks, CA: Sage.
8. O’Connor H, Madge C, Shaw R, et al. (2008) Internet-based interviewing. In: Fielding N, Lee N and Blank G (eds) The SAGE Handbook of Online Research Methods. London: Sage, 271–289.
9. Wittenbaum, G.M. (2000). The bias toward discussing shared information: Why are high status group members immune? Communication Research, 27,379-401.
10. Harris, C. B., Paterson, H. M., & Kemp, R. I. (2008). Collaborative recall and collective memory: What happens when we remember together? Memory, 16(3), 213–230.
11. Storck, J. S. 1995. Oh say, can you see: The impact of video communication on attention, workload, and decision-making. Unpublished doctoral dissertation, Boston University, Boston.
12. Ferran, C., Watts, S., 2008. Videoconferencing in the Field: A Heuristic Processing Model. Management Science 54, 1565–1578.
13. Rettie, R. (2009). Mobile phone communication: Extending Goffman to mediated interaction. Sociology,
14. Deakin, H., & Wakefield, K. (2014). Skype interviewing: Reflections of two PhD researchers. Qualitative Research
15. Duncombe, J., & Jessop, J. (2012). “Doing rapport” and the ethics of “faking friendship”. In T. Miller, M. Birch, M. Mauthner, & J. Jessop (Eds.), Ethics in qualitative research (2nd ed.). London: Sage. Ch. 7.
16. Seitz, S. (2015). Pixilated partnerships, overcoming obstacles in qualitative interviews via Skype: A research note. Qualitative Research, early online publication, 16, 229–235.