How to Diagnose Unsupported LLM Claims: A Grounding Checklist

An LLM can sound certain and still be wrong. The skill that matters is not deciding whether to trust the model wholesale—it is learning to sort each claim into the right category and act accordingly.

Why "Is This a Hallucination?" Is the Wrong Question

You ask a model a question. It answers with confidence, smooth phrasing, and just enough specific detail to feel reliable. Then you check the source material and find that one sentence is accurate, one is subtly wrong, and one is entirely invented. Was that answer a hallucination?

The question sounds reasonable, but it is the wrong unit of analysis. A single response can mix grounded facts, invented details, and correct-but-irrelevant content. Judging the whole answer as "hallucinated" hides what actually went wrong—and judging it as "fine" because most of it checks out lets a fabricated detail slip through.

The useful move is to stop asking whether the answer is a hallucination and start asking what state each claim is in. I use four categories:

• Supported — the claim is present in or directly follows from the provided context.
• Contradicted — the claim actively conflicts with the provided context.
• Missing — the claim is absent from the context but might be true from general knowledge.
• Unverifiable — you have no way to confirm the claim with what you have.

This is the grounding checklist. It replaces vague worry with a concrete diagnostic. If you have read about why hallucinations happen, you already know that generation can outrun evidence. This article is about the inspection step that comes after: checking each claim against the evidence the model was actually given.

The Grounding Checklist: Four Questions for Every Claim

Run this decision procedure on any answer, sentence by sentence or claim by claim. For each claim, ask:

1. Is this claim present in the provided context or source? If yes, it is supported.
2. Does the context directly contradict this claim? If yes, it is contradicted.
3. Is the claim absent from context but plausibly true from general knowledge? If yes, it has missing evidence.
4. Can I verify this at all from what I have? If no, it is unverifiable.

The same claim can land in different categories depending on what context you supplied. Ask the same question with a detailed document as context versus no context at all, and you will get different answers about whether a claim is grounded. That is not a flaw in the checklist—it is the point. Grounding is always relative to the evidence available.

Supported vs. Contradicted: The Two States You Can Actually Decide

These two categories are the most actionable because you can decide them with certainty. You have the context. You can check.

Supported means the claim restates or directly follows from the context. This is the state you want for grounded tasks like summarization, question answering over your documents, or any workflow where the model should stay inside the supplied material. No repair action needed.

Contradicted means the model overrode the context with its own prior knowledge. This is a clear failure signal. Imagine you give the model a document stating that your company's product launched in 2021. The model answers that it launched in 2019—because its training data "knows better." The model did not lack information. It trusted its parametric memory over the source you provided.

When you find a contradicted claim, the fix is usually upstream. This points to a prompt or retrieval problem, not a model defect. The model needs clearer instructions to treat the provided context as authoritative, or the retrieval system failed to surface the right evidence. Check what the model was actually given before you blame generation.

Common mistake: Treating "not in my context" as proof of fabrication. Absence is not contradiction. A claim can be missing from your document and still be true—it is just not grounded there.

Missing and Unverifiable: The Two States That Need Judgment

The harder categories are the ones where you cannot simply check against context. These require judgment about stakes and task requirements.

Missing evidence means the claim is absent from the context but may be true from general knowledge. The model might answer a question about your internal API by adding a general fact about REST conventions that was not in your documentation. That fact could be correct. It is just not grounded in the source you supplied.

The decision rule here depends on your task. If you are building a system where every claim must trace to a source, missing evidence is a failure—the model should have stayed silent or flagged the addition. If you are using the model for brainstorming or general explanation, a plausible claim from general knowledge may be acceptable. The key is knowing which mode you are in.

Unverifiable means you have no context, no reliable source, and no way to confirm. Treat these claims as untrusted by default. In high-stakes tasks—medical advice, financial decisions, legal analysis—an unverifiable claim should trigger a refusal, a caveat, or a source requirement. Silent acceptance is not an option.

Note: "Not in my document" does not automatically mean "false." It means "not grounded here." The distinction matters because it changes your repair action. Missing evidence can be fixed by adding retrieval or source material. Unverifiable claims may need to be rejected outright.

Common Mistakes When Checking Claims

A grounding check is only as good as the discipline behind it. Watch for these recurring errors:

Judging the whole answer instead of individual claims. One bad sentence can poison your verdict on good ones. A response with three supported claims and one fabricated detail is not entirely trustworthy—but it is also not entirely useless. Sort each claim.

Treating absence as fabrication. As noted above, a claim missing from your context is not automatically invented. It may be true from general knowledge. The category matters because the repair action differs.

Trusting plausibility over grounding. A claim that sounds reasonable or matches your general knowledge can still violate the task requirement to stay grounded in the supplied source. Plausibility is not evidence.

Assuming confidence correlates with accuracy. Fluent, detailed, confident phrasing does not indicate reliability. LLMs produce confident falsehoods precisely because their training optimizes for plausible text, not verified truth.

Turning the Checklist Into a Repair Workflow

The four categories only earn their keep when they map to concrete fixes. Here is how I use them in practice:

Contradicted and missing claims often trace back to retrieval gaps or weak context. Before you blame the model, inspect what it was actually given. Was the relevant document retrieved? Was the context window crowded with irrelevant material? Did the prompt instruct the model to stay grounded?

Unverifiable claims in high-stakes output should never pass silently. Build a refusal or caveat into your system. If the model cannot ground a claim, it should say so.

When you see repeated failures in one category, the fix belongs upstream. Frequent contradictions suggest a prompt problem—the model does not understand that the context is authoritative. Frequent missing claims suggest a retrieval problem—the right evidence is not reaching the model. Tighten the retrieval, sharpen the prompt, or add a stricter task contract.

The checklist is lightweight enough to run manually on a few answers. Once you trust it, you can encode it later as an automated grader or evaluation case. That is the path from manual inspection to systematic evaluation.

The Decision Rule

Here is what I want you to do with your next LLM answer: run the four-question checklist on each claim. Sort every claim into its category. Act only on the states that demand repair—contradicted claims need a prompt or retrieval fix, missing claims need better sourcing, unverifiable claims need refusal or flagging in high-stakes work. Supported claims you can trust.

The skill is classification, not suspicion. Knowing which claims to trust is what turns an LLM from a confident guesser into a usable tool.