The same question never gets the same answer twice

Type the same question into the same model twice and you will get two different answers. Usually they agree. Sometimes they do not.

This surprises people, and it is worth understanding, because it explains a whole class of strange behaviour.

What is happening

When a model produces a response, it does not look up an answer. It works out, for each step, how likely each possible next piece of text is, then picks one.

If it always picked the single most likely option, every answer to the same question would be identical. So instead it samples — with a bias toward the likely options, but not a rule.

That is the source of the variation. It is a deliberate design choice, and it is the same mechanism that lets a model be asked for ten different headlines without producing the same one ten times.

Why it is not simply a flaw

A model that always produced the single most likely continuation would be repetitive and dull, and worse at anything creative. Ask for five names for a product and you would get one name five times.

The randomness is a feature with a cost. The cost is that answers are not reproducible.

What it means in practice

Accuracy varies run to run. A model that gets a question right nine times in ten will get it wrong on the tenth attempt. The failure is not a different model; it is the same model, sampled differently.

An answer is not evidence. Getting a good answer once does not mean you will get it again. If something must be right, ask more than once and compare.

Some tools reduce it. Most APIs let you lower the temperature — the setting that controls how adventurous the sampling is. At its lowest the model becomes nearly deterministic, and nearly repetitive.

It makes reproducibility hard. In any serious application this matters: the same input producing different output is a property most software does not have, and systems built on AI have to be designed around it.

What to do about it

Three habits that hold up:

1. Ask twice when it matters. Disagreement between two answers is a useful warning that the question is in uncertain territory. 2. Lower the temperature for factual work, if your tool exposes the setting. Determinism is worth more than variety when you want a correct answer. 3. Keep the version you checked. If you verified something, save the text. Re-asking does not reproduce it.

The thing to hold on to

A language model is not a database that returns a stored fact. It is a process that produces plausible text, and it runs slightly differently each time.

That is what makes it useful, and it is exactly why the output needs checking rather than trusting.