How to check what it tells you

A model is fluent before it is correct, and there is no reliable signal in the text itself that tells you which you are getting. So the skill is not reading the answer. It is checking it.

Why the usual instinct fails

Most people check AI output by asking "does this sound right?"

That test does not work, because sounding right is exactly what the model is good at. Text quality and truth quality come from the same mechanism and arrive together.

Worse, the failures cluster in the places that look most authoritative: specifics. A general explanation is usually fine. The invented part is the date, the quotation, the citation, the number.

The four checks

1. Does it contain anything specific?

Dates, names, numbers, quotations, citations, URLs. Find them, then verify them independently. Treat every specific as unverified until you have.

2. Are the citations real?

This is the classic trap. Asked for sources, models will produce references that look perfect — correct-looking authors, journals, years — for papers that do not exist. A fabricated citation is worse than no citation, because it is designed to satisfy you.

Open the link. Every time. If there is no link, there is no source.

3. Does it contradict itself?

Ask the same question a second way. Ask it to state its confidence. Ask what would change its mind. Re-ask in a fresh conversation. A claim that shifts under any of those is not one to rely on.

4. Can it be true?

Sanity-check the shape of the claim: is that number plausible? Is that date possible? Is that person old enough to have done that? This catches a surprising amount, and it costs nothing.

The test that builds calibration fastest

Ask it about something you are an expert in.

Your job, your hobby, the subject you studied. You will find the answer is 85% right, fluently phrased, with one or two confident errors. Knowing that ratio changes how you read everything else it produces.

Habits worth keeping

• Ask for uncertainty. "Which parts of that are you least sure about?" — it often flags its own weak spots, if asked.
• Ask for sources, then check them. Unchecked sources are decoration.
• Use it on text you provide, where hallucination is structurally harder.
• Never let it be the only reader. For anything that matters, a human checks the result. That is not a limitation of the current models; it is the shape of the tool.
• Keep the version you checked. If you verified something, save it. The model will not produce the same answer tomorrow.

A note on what this means for trust

This is not a reason to distrust AI in particular. It is a reason to be specific about trust in general — the same discipline you would apply to a newspaper, a Wikipedia article, or a confident colleague.

The difference is that AI produces its output at a speed and in a volume that makes checking feel unnecessary. It is necessary. That is the whole lesson.

Where to go next

• What it can and cannot do — where the failures cluster
• Prompting that actually changes the output
• Benchmarks, and why the numbers lie