Accuracy & Hallucination

The Fabrication Hunt

Deliberately induce a confident, wrong answer, then find the prompt wording that prevents it. The most useful twenty minutes you can spend on calibrating how much to trust a model.

Self-consistency

Self-consistency runs the same reasoning prompt several times at non-zero temperature and takes the most common answer. It trades cost for accuracy on problems with a single verifiable result.

Prompt chaining

Prompt chaining splits a complex task into a sequence of focused prompts, where each step's output feeds the next. Chains are easier to debug and more reliable than one large prompt.

Grounding

Grounding means tying model output to verifiable sources — retrieved documents, search results, or database records — rather than relying on parametric memory. Grounded answers can be checked.

AI agent

An AI agent is a system where a model plans and takes actions through tools in a loop, rather than producing a single response. It decides what to do next, does it, observes the result, and continues until the goal is met.

Hallucination

A hallucination is model output that is fluent and confident but factually wrong — invented citations, non-existent functions, fabricated statistics. It stems from models being trained to produce plausible text, not verified truth.

Prompt injection

Prompt injection is an attack where malicious instructions hidden in content the model processes override the developer's intended behaviour. It is the most serious unsolved security issue in LLM applications.

Lost in the middle

Lost in the middle is the tendency of models to use information at the start and end of a long context well while under-weighting material in the middle. Position affects whether the model actually uses what you provided.

Model drift

Model drift is when a hosted model's behaviour changes over time as the provider updates it, causing prompts that worked reliably to degrade without any change on your side.

Confidence score

A confidence score estimates how reliable an output is. Language models do not natively produce calibrated confidence, and asking a model how sure it is yields poorly calibrated self-assessment.

Bias

Bias in AI is systematic skew in outputs that reflects patterns in training data or training process — including demographic stereotypes, cultural assumptions, and over-representation of dominant viewpoints.

Citation

A citation links a model claim to the source it came from. Real citations come from grounding in retrieved documents; models asked to cite from memory frequently invent plausible-looking references.

Fact-Check This Claim

Verifying a statistic or claim before publishing. Run this on anything you did not source yourself, especially numbers pulled from a chat model — statistics are exactly the kind of detail models hallucinate confidently. Use a search-grounded tool so you get a live primary source and a date, and treat "no source found" as a reason to cut the claim rather than to keep looking.