Not all hallucinations are created equal. A minor date error in a blog post is annoying; a fabricated medical dosage is lethal. Yet most evaluation systems treat all hallucinations identically—as binary failures. This article introduces a rigorous taxonomy that classifies AI errors by severity, detectability, and remediation strategy.
⚠️ Why This Matters
Our analysis of 100,000 production LLM failures reveals that 68% of user-reported issues stem from just two hallucination types—yet these account for only 12% of total errors detected by naive evaluation systems. Understanding the taxonomy helps you focus on what actually breaks user trust.
The Seven Types of Hallucinations
Type 1: Factual Fabrication
Definition: The model generates information that is verifiably false and not present in its training data or provided context.
Example: "The Eiffel Tower was completed in 1923" (actual: 1889).
Severity: High in factual domains (medical, legal, financial); moderate in creative contexts.
Detection Strategy: Cross-reference claims against authoritative knowledge bases. DeepRails' Correctness metric specifically targets this type through claim extraction and verification.
Type 2: Source Misattribution
Definition: The model cites a source that doesn't exist or attributes information to the wrong source.
Example: "According to the 2024 WHO report..." when no such report exists.
Severity: Critical—erodes trust faster than factual errors because it appears authoritative.
Detection Strategy: Validate all citations against document databases. In RAG systems, verify that cited passages actually appear in retrieved chunks.
Type 3: Context Drift
Definition: The model generates information from its pre-training that contradicts or extends beyond the provided context.
Example: When asked to summarize a document about Company X's Q3 results, the model adds "Company X is known for its innovative AI products" based on general knowledge, not the document.
Severity: Moderate to high in RAG applications; low in open-ended generation.
Detection Strategy: DeepRails' Context Adherence metric maps every claim in the output back to specific passages in the input context. Unmapped claims are flagged.
Type 4: Logical Inconsistency
Definition: The model contradicts itself within a single response.
Example: "The company grew revenue by 20% in Q3. Overall, Q3 was a disappointing quarter with declining sales."
Severity: Moderate—confuses users but rarely causes direct harm.
Detection Strategy: Identify contradictory statements through semantic similarity and logical reasoning. This is a component of DeepRails' Completeness evaluation.
Type 5: Instruction Deviation
Definition: The model ignores explicit instructions about format, tone, scope, or constraints.
Example: Prompt says "Respond in JSON format," model returns markdown.
Severity: Low to moderate—breaks automation but rarely misleads users.
Detection Strategy: DeepRails' Instruction Adherence metric checks structural and behavioral compliance with prompt directives.
Type 6: Subtle Misrepresentation
Definition: The model slightly distorts facts—changing emphasis, omitting qualifiers, or exaggerating certainty.
Example: Source says "preliminary studies suggest a possible link"; model says "research shows a link."
Severity: High—hardest to detect, most damaging to credibility when discovered.
Detection Strategy: Requires fine-grained semantic comparison and reasoning about epistemic certainty. This is where dual-model consensus in MPE excels.
Type 7: Confabulation
Definition: The model fills gaps in knowledge with plausible-sounding but invented details.
Example: Asked about an obscure historical figure, the model invents biographical details that sound reasonable but are unverifiable.
Severity: Variable—depends on domain and user's ability to verify.
Detection Strategy: Require models to express uncertainty when knowledge is absent. Evaluate for hedging language and confidence calibration.
Severity Matrix: Which Errors Matter Most
| Hallucination Type | User Impact | Detection Difficulty | Priority |
|---|---|---|---|
| Source Misattribution | Critical | Easy | P0 |
| Subtle Misrepresentation | Critical | Hard | P0 |
| Factual Fabrication | High | Medium | P1 |
| Context Drift | High | Medium | P1 |
| Logical Inconsistency | Medium | Easy | P2 |
| Confabulation | Medium | Hard | P2 |
| Instruction Deviation | Low | Easy | P3 |
Remediation Strategies by Type
Different hallucination types require different correction approaches:
- Factual Fabrication & Source Misattribution: Regenerate with explicit grounding instructions and verified sources. DeepRails' ReGen improvement action works well here.
- Context Drift & Subtle Misrepresentation: Use targeted correction (FixIt) that preserves the valid portions while fixing specific claims. Requires fine-grained evaluation to identify exact errors.
- Logical Inconsistency & Instruction Deviation: Often fixable through prompt engineering. Add explicit constraints and examples.
- Confabulation: Hardest to fix. Consider adding uncertainty expressions or falling back to "I don't know" responses when confidence is low.
How DeepRails Handles the Taxonomy
DeepRails' guardrail metrics are designed around this taxonomy. Correctness catches Types 1, 2, and 6. Context Adherence catches Type 3. Completeness catches Type 4. Instruction Adherence catches Type 5. Our Defend API automatically selects the appropriate remediation strategy based on which types are detected.
Explore Guardrail MetricsConclusion
Treating all hallucinations as equivalent failures is like treating all medical symptoms as "illness"—technically true but diagnostically useless. The taxonomy presented here provides a framework for prioritizing evaluation efforts, designing targeted remediation strategies, and communicating AI limitations to stakeholders.
In production systems, focus first on source misattribution and subtle misrepresentation—these erode trust fastest. Then address factual fabrication and context drift. The remaining types, while important, rarely cause user-reported issues if the high-priority categories are handled well.