RAG Hallucinations: Why RAG Still Makes Things Up, and How to Catch It

Yes, RAG still hallucinates. Retrieval-augmented generation (RAG) hands the model relevant documents, but nothing forces the answer to stay inside them: the model can contradict a passage, add claims no passage supports, or cite a source that says something else. Stanford researchers found that legal research tools built on RAG still gave hallucinated answers 17% to 33% of the time, so a RAG app needs a check on every answer, not only on retrieval.

This guide covers the four ways a RAG answer goes wrong, why an answer grounded in your documents can still be wrong, the RAG evaluation metrics that measure each failure, and how to catch hallucinations before users see them.

Does RAG stop hallucinations?

No. RAG reduces hallucinations; it doesn't remove them. In the first preregistered study of commercial legal AI tools, Varun Magesh, Daniel E. Ho and colleagues at Stanford tested LexisNexis's Lexis+ AI and Thomson Reuters' Westlaw AI-Assisted Research and Ask Practical Law AI, all of which use RAG. Each hallucinated between 17% and 33% of the time: less often than GPT-4 on its own, but far from never (Magesh et al., "Hallucination-Free?", May 2024).

RAG errors are also harder to spot than a plain chatbot's, because they come wrapped in real sources. The Stanford team named one pattern "misgrounded": the answer cites a real document, but the document doesn't support the claim.

The four ways a RAG answer goes wrong

A RAG answer goes wrong in four ways: it contradicts the source, adds what the source doesn't say, cites a source that doesn't back it, or answers from the wrong document. The first two come from RAGTruth, a set of nearly 18,000 RAG answers that researchers labeled word by word (Niu et al., 2024). The last two come from the Stanford legal study.

1. It contradicts the source

The answer changes what a passage says. In one RAGTruth example, the source said the best pictures come "between 24 and 30 weeks" of pregnancy; the answer said "between 20 and 32 weeks." RAGTruth calls a changed number or name an evident conflict, and a swapped word that shifts the meaning or severity a subtle conflict.

2. It adds what the source doesn't say

The answer goes beyond the documents with an invented detail, or with an inference stated as fact. A policy says refunds take 10 business days; the answer says they take 10 business days "and usually arrive sooner." RAGTruth calls this introducing baseless information, which includes "inferred details, insights, or sentiments" the source never gave.

3. It cites a source that doesn't back the claim

The citation is real, but the cited passage says something else. The Stanford study calls this a misgrounded answer; when the cited source doesn't exist at all, it calls the answer fabricated. Both are dangerous because a citation makes an answer look checked. The seven types of AI hallucinations cover it as source misattribution.

4. It answers from the wrong document

Retrieval returns a passage that is out of date, about the wrong product or jurisdiction, or off topic, and the model answers from it faithfully. The Stanford team counted citing an inapplicable source as misgrounded too. Long inputs add to the risk: models use information best at the start or end of a long context and noticeably worse in the middle (Liu et al., "Lost in the Middle", 2023).

Grounded isn't the same as correct

A RAG answer can be grounded in your documents and still be wrong, or correct and not grounded. Measure the two separately and you know what to fix.

• Context adherence (also called groundedness or faithfulness) asks whether every claim traces back to the documents the model was given.
• Correctness asks whether every claim is true, wherever it came from.

DeepRails' documentation gives the classic case: "The Eiffel Tower is in Paris" is correct even when the documents never mention it, but it scores low on context adherence if the model was told to answer only from the documents (Correctness docs). Together the two measures give four outcomes:

An answer that contradicts its own source can be costly. Air Canada's website chatbot told a customer they could claim a reduced bereavement fare after travel, within 90 days of the ticket's issue date. The chatbot linked to the airline's own "Bereavement travel" page, which said the policy did not apply once travel was complete. In February 2024, British Columbia's Civil Resolution Tribunal held the airline responsible: "It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot" (Moffatt v. Air Canada, 2024 BCCRT 149). An answer that contradicts the page it links to is exactly what a grounding check looks for.

Which measure matters more depends on the job:

• Context adherence first: legal and compliance answers, customer support that must follow official policy, and summaries of a specific document, where anything added from outside is a liability.
• Correctness first: medical and educational content, where a false statement does harm whatever its source.
• Both: financial reporting and other regulated work, which needs answers that are true and traceable to a source.

RAG evaluation metrics: what to measure

Measure retrieval and the answer separately. Retrieval metrics tell you whether the right passages came back. Answer metrics tell you whether the model stayed inside them, told the truth and answered the whole question.

Most of these metrics break the answer into separate claims and check each one. Ragas, for example, scores Faithfulness as the number of claims the retrieved context supports divided by the number of claims in the answer (Ragas docs). TruLens calls context relevance, groundedness and answer relevance the "RAG triad" (TruLens docs). DeepRails' Context Adherence matches each claim to the context, gives a yes-or-no verdict with a confidence rating, and combines them into a score from 0 to 1 (Context Adherence docs). The guide to LLM evaluation metrics covers the rest of the set.

Two cautions. First, use context adherence only when the prompt carries real context: by design, DeepRails' check stops early and scores 100% when it finds too little context to judge. Second, don't let a model grade its own answer, because it shares the blind spots that produced the error (see LLM as a judge). DeepRails judges each claim with two different models, and the model that wrote the answer never judges it (how the evaluation engine works).

How to catch RAG hallucinations before users see them

Score every answer on context adherence and correctness, and fix or hold any answer below your threshold before it is shown. Five steps get you there:

1. Clean the source. A perfectly grounded answer from a wrong or stale document is still wrong. Date your documents and retire old versions.
2. Tell the model its limits. Instruct it to answer only from the passages provided and to say when they don't contain the answer.
3. Check answers before delivery where mistakes are costly. A guardrail scores each answer and compares each score with your threshold. In DeepRails Defend, an answer below threshold is repaired (FixIt), written fresh (ReGen) or logged without change (Do Nothing), and a repaired answer is scored again before it is delivered. By default Defend allows three attempts in all, the first answer plus two repairs; you can set from one to five (Defend docs).
4. Monitor everything else. Where a delay isn't acceptable, score answers after they are sent and watch the failure rate over time (LLM monitoring and drift detection).
5. Test the guardrail itself. Run it on answers you know are wrong and on answers you know are right. It should catch the first without flagging the second; adjust thresholds until it does.

Fixing usually serves users better than blocking. A blocked answer ends the conversation; a fixed one lets it continue.

Key takeaways

• RAG reduces hallucinations but doesn't end them: commercial legal tools built on RAG hallucinated 17% to 33% of the time in Stanford's study.
• RAG answers fail in four ways: they contradict the source, add unsupported claims, cite sources that don't back them, or answer from the wrong document.
• Grounded and correct are different measures. Score both, and you know whether to fix the model, the instructions or the documents.
• Measure retrieval (context relevance and recall) separately from the answer (context adherence, correctness and completeness).
• Check answers before delivery where a wrong one is costly, and monitor the rest.