LinkedIn: The Hallucination Problem

"ChatGPT told my customer our Pro plan costs $29. It's $25."

That happened to one of our early users. Sounds minor — until the customer replies asking why your pricing page says one thing and your email says another. Now you're explaining a $4 discrepancy instead of closing the deal.

And it's not a one-off.

In our survey of 200+ Gmail power users, 67% said they'd sent an AI-generated email with incorrect information. For most of them, AI didn't remove work. It moved the work — to fact-checking every draft before hitting send. If you're still doing that, your "AI assistant" is just a slightly faster text editor.

The confidence problem

I've done this myself. Sent a draft without double-checking the numbers because the tone sounded right. The confidence of the writing masked the wrongness of the content.

That's the specific failure mode generic AI creates in email: it doesn't signal uncertainty. It doesn't say "I'm not sure about this pricing detail." It generates fluent, well-phrased text and lets you assume it's accurate. The mistake feels like yours when the customer catches it — but the AI handed you a draft that looked ready to send.

This is especially costly in the moments that matter most. A pricing question on the verge of closing. A billing dispute that needs a careful, accurate response. A support escalation where the customer is already frustrated. Generic AI is worst exactly when the stakes are highest, because high-stakes emails are the ones that depend on specific, verifiable business details the model has no access to.

That's not an AI problem. It's a grounding problem.

Generic AI writes from what it thinks it knows — a statistical average of similar-sounding businesses. Grounded AI writes from your actual pricing page, help center, docs, and past conversations.

That doesn't make it perfect. But it makes it verifiable.

Here's what that looks like in practice. A customer asks about your Pro plan:

Generic AI draft:

"Our Pro plan is $29/month and includes advanced features, priority support, and enhanced analytics."

Confident. Clean. Wrong on the price, vague on the features.

Grounded AI draft:

"The Pro plan is $25/month (or $20/month billed annually). It includes 500 AI drafts, all KB integrations — Google Drive, Notion, Confluence, Dropbox — priority support, and custom tone settings." Source: pricing-page.md, updated Jan 2025

The difference isn't the writing quality. It's that the second draft cites exactly where the answer came from. You can verify the claim in five seconds instead of opening three tabs to check.

That's the shift: from "does this sound right?" to "I can confirm this is right."

Why grounding is harder than it sounds

Connecting AI to your docs isn't plug-and-play. The hard part isn't retrieval — it's reliable retrieval.

Keyword search misses intent. A customer asks "what happens to our data if we cancel?" Your docs say "retention and deletion policy." A keyword search finds nothing relevant. Meanwhile, the model fills the gap with something plausible.

Semantic search finds passages that feel related — but related isn't the same as accurate enough to send to a customer. A passage about enterprise security can look relevant to a data retention question. A roadmap note can match a feature question even though it describes something that hasn't shipped yet.

Good grounding requires a full pipeline: retrieve broad candidates, filter for evidential strength, re-rank for answerability, and only pass passages that can actually support a factual claim into the draft. Then attach the source to each claim so the person reviewing can verify before sending.

That's what we built at Inbox SuperPilot. It connects to 20+ sources — Google Drive, Notion, your help center, CRM data. It creates drafts for your review, not auto-sends. And it shows you exactly where each claim came from, so the review step takes seconds instead of minutes.

The real cost of ungrounded AI

The $4 pricing error above got caught. Many don't.

A wrong refund policy quoted in writing. An integration promised that's on your roadmap, not your product. A contract term paraphrased incorrectly to a client who has the original in front of them. These aren't embarrassing footnotes — they create precedent, erode trust, and generate cleanup work that costs far more than the time the AI saved.

At 20 emails a day, even three minutes of fact-checking per draft is an hour of verification work daily. That's not productivity. That's latent cost — invisible until someone catches the mistake.

Grounded AI with source citations doesn't eliminate review. You still read every draft before sending. But it changes what review feels like: from anxious line-by-line checking to a quick confirm against a cited source. The cognitive load drops, the trust goes up, and the mistakes that do slip through are far easier to catch.

Verifiable beats confident-sounding

The AI email market is splitting into two camps. Writing assistants that help with tone, length, and fluency — and knowledge-grounded systems that actually know your business.

For low-stakes internal emails, the first camp is fine. For anything customer-facing that involves your pricing, policies, product details, or past commitments, the second camp is the only one that doesn't create new risk while reducing old effort.

Grounded AI won't write a perfect email every time. But it will write a verifiable one — and that's the difference between a draft you can confidently send and one you have to fact-check before you do.

If you want to see what grounded drafts look like in your inbox, Inbox SuperPilot is free to start — no card required.

What's the most expensive AI hallucination you've had to clean up? We'd genuinely like to know — it shapes what we build next.

Further Reading & References

From the Inbox SuperPilot Blog

• What We Learned Building a Citation Engine for Email AI
• Gmail Gemini vs. Knowledge-Grounded AI: An Honest Comparison
• Why Generic AI Fails in Customer Support Email Workflows
• ChatGPT vs. Gemini vs. Claude for Email: Where Generic AI Falls Short

External References

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis et al., 2020. The foundational paper on why grounding generation in retrieved documents improves factual accuracy.
• RAGAS: Automated Evaluation of Retrieval Augmented Generation — Es et al., 2023. Framework for evaluating retrieval quality and answer faithfulness.
• Factuality Challenges in the Era of Large Language Models — Augenstein et al., 2023. Survey of hallucination types and mitigation strategies in LLMs.
• Lost in the Middle: How Language Models Use Long Contexts — Liu et al., 2023. Why evidence position in retrieved context affects whether models actually use it.