The verification habit — telling when Claude is wrong

Working with Claude — CC BY 4.0

Lesson 1.4 changed what Claude says: label each claim, cap the guesses, own the rule-breaking. This lesson is about what you do with what comes back. Because the machine mislabels as easily as it misstates — a claim tagged [KNOWN] can still be wrong. The tags don’t make Claude reliable. They make it checkable. Verification is the habit that turns “checkable” into “checked”.

The whole point of the course sits here. Not going faster — being able to rely on the output. That only happens if the last set of eyes on an answer is yours.

Start by reading the tags, not the tone

Claude sounds equally fluent whether it’s right or wrong. Fluency is not evidence. So before you react to how an answer reads, look at the labels you asked for in 1.4:

• Anything tagged [GUESS], or a [FRAME] claim dressed up as real-world advice — treat as a lead to check, never as a finding.
• [INFERRED] — a chain of reasoning. Chains break. Check the links, not just the conclusion.
• [KNOWN] and [COMMON] — usually solid, but these are exactly the claims that get stated with most confidence and least hedging, so they’re where a wrong one does the most damage.
• [COMPUTED] — re-run the sum yourself. Language models are shaky at arithmetic, and a wrong figure arrives looking as tidy as a right one.

If a claim carries no tag and it matters, that’s your signal to ask for one before you go further.

Ask for the source, then actually open it

For anything you’ll rely on — a law, a dosage, a citation, a named person, a figure you’ll put in front of someone else — ask plainly:

What’s your source for that? Give me the specific document or page, not a general area.

Two things happen. Sometimes Claude can’t produce one, which tells you the claim was built from pattern rather than fact — downgrade it. Sometimes it produces a source that, when you open it, says something different. Never invent citations was a rule in 1.4; this is you enforcing it. A source you haven’t opened is not really a source, just a claim about one.

If web search is switched on in the Claude app, Claude can look things up live and cite the URLs it used, and you can click straight through to check them. (Web search is a toggle in the app; confirm its exact position and behaviour in-app or in current docs, as the interface shifts between updates.) Note the difference: with search on, you get real links to open; working from training memory alone, you get recollection, which is where invented citations come from. When it matters, prefer the version you can click.

Spotting confident-but-wrong

The dangerous answer is the confident-sounding one — smooth, specific, authoritative, and false. A few tells to learn:

• False precision. Exact dates, section numbers, percentages and quotes that feel too neat. Specificity is easy to generate and reads as authority. Verify the specifics first — they’re the most quotable and the most likely to be confidently wrong.
• One tidy theory that explains everything. Real situations are messier. If the answer is suspiciously elegant, that’s a prompt to probe, not to relax.
• Instant caving, or instant fighting. If Claude flips its position the moment you push back — or argues on reflex when you’re plainly right — the position was never anchored to anything. Ask why it changed. “No real reason” is your answer.

A quick cross-check: ask the same question a second way, or in a fresh conversation with no leading framing, and see if the answer holds. If the two versions disagree, at least one is wrong and you’ve found the seam to dig into.

The “would this have predicted it in advance” test

This one is worth keeping for the rest of your life, well beyond Claude. When you’re handed an explanation — why a market moved, why a person behaved as they did, why a project failed — ask:

Would this account have predicted the outcome before it happened, or does it only fit now that we know how it turned out?

An explanation that only clicks into place after the fact predicts nothing; it fits the story to the result. In 1.4 you told Claude to flag these as [INFERRED, post-hoc] — explains but doesn’t predict. Here you apply the same test to everything it hands back, and to your own reasoning while you’re at it. Hindsight dressed as insight is the most comfortable error there is.

Where the habit lives

You don’t run the full drill on every reply — that would be exhausting and pointless for low-stakes chat. You run it in proportion to what a wrong answer would cost. Casual question, glance at the tags and move on. Something you’ll act on, spend money on, or put your name to — check the source, re-run the numbers, cross-check once, apply the hindsight test.

That proportionality is the skill. Claude drafts; you decide what’s true. The output is yours — you directed it, you reviewed it, you own it. A tool you can rely on isn’t one that’s never wrong. It’s one you’ve built the habit of catching.