Article by ICG member Arthur Fletcher
Imagine this: You have conducted a short online research community and you have a lot of comment to analyse and report on (20 participants, 30,000 words in all). You have been offered the use of AI to assist with your analysis and reporting.
The AI supplier has told you:
- The AI takes everything into account, all comments from all participants.
- The AI down-weights the comments from moderators. The moderator probably comments more than any single participant in the community and their words are often important to understand the context of some participant replies (but their peripheral comments will be ignored)
- The AI is fully briefed on the project background and objectives – taken from your proposal – and anything else you choose to include. Each community task has its own specific objectives. The AI can also reference specific, trusted websites for additional market/category/product/brand context
- The AI can analyse by each task in your community as well as groups of tasks (as a combined topic) or analyse all tasks at once
- The AI can provide summaries, highlight key topics, identify consumer segments and recommend key comments to incorporate into your report
- The AI can be asked to write in a chosen style – scientific, contemporary, first person etc., you can also upload previous reports that you have written so that it mirrors your own style
- The AI can also draw conclusions, identify insights and make recommendations based on the briefing
- The AI can write your final report, providing links to the relevant source material
So, you go for it (hand over some old reports/docs you have written) and you have the community report in your hands within an hour.
Of course you are going to read the report and tune the language to your requirements. You have done the moderation for the community so can review the content and do a memory sense check alongside any contemporaneous notes you’ve made. You will also review the conclusions, insights and recommendations against the project objectives and see if you agree with them.
But are you going to do any more checks? If so, how? Do you go back to the source material? Do you review all the comments or just some? If you read them all, what’s the point of using the AI if you have to go back through everything? If the first report is good, would you still want to review every subsequent AI generated report in the same way?
Is the AI report better than your own reporting? The AI uses a large language model (LLM) which is trained to generate natural text in a style of your choosing, it has a large vocabulary and can take everything you upload into account. You can also choose how long your report is going to be. If reporting is a chore for you, is AI likely to do a better job than you?
Would you tell your client? What difference would it make to them if they knew your report was (primarily) AI generated? Would they consider it better or worse? Would (future) professional standards or legislation oblige you to tell your clients?
Would you charge the same? An AI report should use less of your time, so would you now have a 2 tier reporting charge – AI (cheaper) or Human (more expensive)?
In the end I don’t think AI removes the need for us – it changes it. Reporting becomes less about writing and more about curating, challenging and validating what the machine produces. That may sound like less work, but in truth it asks us to be more thoughtful than ever. The question is: are we ready to step into that role or will we let the machine define the story for us?