Core concepts
Temperature
Definition
Temperature controls how random a model's word choices are. Low values (0-0.3) make output focused and repeatable; high values (0.8-1.2) make it more varied and creative but less reliable.
At each step a model produces a probability distribution over possible next tokens. Temperature reshapes that distribution before a token is sampled. At temperature 0 the model always takes the most likely option, making output nearly deterministic. Raising it flattens the distribution so less likely tokens get a real chance.
Choose by task, not by taste. Extraction, classification, code generation and anything a program will parse want low temperature — you want the same input to give the same output. Brainstorming, creative writing and generating multiple distinct options want higher temperature.
A common mistake is raising temperature to fix boring output. Usually the problem is a vague prompt, not insufficient randomness. Fix the prompt first; higher temperature applied to a vague prompt just produces varied vagueness.
Related terms
Top-p (nucleus sampling)
Top-p limits the model to choosing from the smallest set of tokens whose combined probability exceeds p. At top-p 0.9, the model samples only from options making up the top 90% of probability mass, ignoring the unlikely tail.
System prompt
A system prompt is a standing instruction that shapes a model's behaviour across an entire conversation, separate from individual user messages. It typically defines persona, rules, tone and output format.
Determinism
A deterministic system returns identical output for identical input. Language models are only approximately deterministic even at temperature 0, because floating-point and infrastructure variation introduce small differences.
Hallucination
A hallucination is model output that is fluent and confident but factually wrong — invented citations, non-existent functions, fabricated statistics. It stems from models being trained to produce plausible text, not verified truth.
Token
A token is the unit of text an AI model reads and writes — roughly four characters or three-quarters of a word in English. Models price by the token and cap context by the token, so token count determines both what you pay and what fits.
Tokenizer
A tokenizer is the component that splits text into tokens before a model processes it. Different model families use different tokenizers, which is why the same text can have different token counts on GPT versus Claude.
Put this into practice
Understanding the term is step one. Our free courses and tools let you actually use it.