--- title: "What Gets Lost in a Convincing AI Answer? | antedote" description: "AI can help teams consider more material and more ideas. The risk is a clear answer that drops the differences a decision should turn on." url: "https://antedote.com/thinking/what-gets-lost-in-a-convincing-ai-answer/" published: "2026-10-22" updated: "2026-10-22" author: "Adam French" --- # What gets lost in a convincing AI answer? AI can help teams explore and interpret more. The risk is losing the differences that should change the decision. Article on AI and judgment, 4 min read Reviewed by [Adam French, Founding Partner](https://antedote.com/why-antedote/#partners) on October 22, 2026. Published October 22, 2026. A team with a decision to make needs a clear answer, and AI can produce one quickly. The question is what it left out. In research, similar words can hide different reasons. In innovation, concepts that look different can rest on the same assumption. Lose those differences, and a business may choose from fewer possibilities than it realizes. The help is real. In a field experiment with 776 professionals at Procter & Gamble, people working alone with AI on real product innovation challenges matched the performance of two-person teams working without it (Dell’Acqua and colleagues, 2025). The experiment judged the work people produced; it couldn’t say whether any of it would have sold. So the case for using AI is strong. What a clear, plausible output can’t show on its own is whether the differences that matter survived. We call the risk flattening: reducing meaningful differences until distinct people, ideas or circumstances look interchangeable. It works differently in research and in innovation, so it helps to keep the two apart. ## Same words, different reasons In research, the risk is a summary that groups people by what they said when the decision depends on why they said it. Imagine three customers who each call a new offer “too complicated”. The first doesn’t understand how it works. The second understands it and can’t see how it would fit into her week. The third could learn it, and doesn’t think it’s worth the effort. A summary that says customers find the offer too complicated is accurate. It also describes three different problems. The first calls for a clearer explanation, the second for a product that fits the way people live, and the third for a stronger reason to bother, which may mean a different proposition or a different audience. Simplify the product, and the business may have answered one of the three. There’s a reason to watch for this in AI summaries in particular. Peters and Chin-Yee tested ten widely used language models on 4,900 summaries of scientific texts. Even when asked for accuracy, most models stated results more broadly than the original texts did. In that study, their summaries were far more prone than expert-written ones to generalize beyond the evidence (an odds ratio of nearly five). Those were scientific papers. That the same tendency applies to summaries of customer research is our inference, and it’s the reason we read an AI summary for what it may have merged. ## Many concepts, one assumption In innovation the risk is different. A long list of polished concepts can vary in name, format and tone while resting on one assumption. Say a team generates forty concepts for a snack brand. They differ in flavor, pack and claim. Look closer and all forty assume the same buyer, eating at the same moment, buying in the same aisle. The list looks wide. The choice it offers is narrow, because every option would succeed or fail for the same reason. Meincke, Nave and Terwiesch reanalyzed published brainstorming experiments, on tasks such as designing toys from everyday objects, in which ChatGPT had made individual ideas more creative. The pools of ChatGPT-assisted ideas were less varied than those from people working without it. The original authors have replied, and both sides accept that the individual ideas improved. These were brainstorming tasks in an experiment. They don’t show company pipelines converging, and we wouldn’t claim they do. What they show is that better individual ideas and a wider set of ideas are separate things, and a team can gain one while losing the other. ## Deciding which differences matter None of this is an argument against summarizing. Synthesis is the job: nobody can act on forty transcripts or two hundred ideas as they stand. And not every unusual observation deserves weight. One striking remark isn’t a segment, and an odd concept isn’t a strategy because it’s odd. Merging the three “too complicated” customers would be fine if a single change would answer all three. The judgment is in knowing which differences would change what the business does, and keeping those. That discipline matters whoever does the summarizing. Human teams also prefer a neat story and a familiar idea, and a confident debrief can flatten a picture as thoroughly as software can. So our concern is the quality of the judgment, whether a person or a model wrote the first draft. Used that way, AI widens what a team can consider: more of the material in view, more possibilities on the table, a first draft sooner. Responsibility for what the differences mean, and for the choice, stays with the people making it. If a decision of yours depends on a summary that feels a little too tidy, we’d be glad to look at it with you. [How we use AI in our own work](https://antedote.com/what-we-do/ai-in-our-work/) ## Sources 1. Dell’Acqua, F., Ayoubi, C., Lifshitz, H., Sadun, R., Mollick, E., Mollick, L., Han, Y., Goldman, J., Nair, H., Taub, S. and Lakhani, K., “[The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and Expertise](https://www.nber.org/papers/w33641),” NBER Working Paper 33641 (April 2025). Working paper; a pre-registered field experiment with 776 professionals at Procter & Gamble, on product innovation challenges. 2. Peters, U. and Chin-Yee, B., “[Generalization bias in large language model summarization of scientific research](https://doi.org/10.1098/rsos.241776),” *Royal Society Open Science* 12(4): 241776 (2025). Peer-reviewed; ten models, 4,900 summaries of scientific texts. 3. Meincke, L., Nave, G. and Terwiesch, C., “[ChatGPT decreases idea diversity in brainstorming](https://doi.org/10.1038/s41562-025-02173-x),” *Nature Human Behaviour* 9: 1107 to 1109 (2025). Peer-reviewed reanalysis of Lee and Chung’s experiments (*Nature Human Behaviour* 8: 1906 to 1914, 2024), published with the original authors’ reply. ## Read next - *Article* · 3 min read ### [Would this grow the business, or just the range?](https://antedote.com/thinking/would-this-grow-the-business-or-just-the-range/) An idea can appeal to customers and still add little to the business. Its value depends on what it changes across the whole offer. - *Article* · 3 min read ### [Does the brand need a new message, or a new direction?](https://antedote.com/thinking/does-the-brand-need-a-new-message-or-a-new-direction/) A new campaign can improve how a brand is understood. Sometimes the harder question is what it needs to stand for. ## How we help [AI in Our Work](https://antedote.com/what-we-do/ai-in-our-work/) How AI helps our researchers search, compare and check, and what our people decide. ## Working through this question now? Bring us the version your team is facing; half-formed is fine. We’ll say how we would approach it, and whether we’re the right fit. [Talk to us](https://antedote.com/talk-to-us/) · [hello@antedote.com](mailto:hello@antedote.com) A partner or director will reply.