Core concepts
Top-p (nucleus sampling)
Definition
Top-p limits the model to choosing from the smallest set of tokens whose combined probability exceeds p. At top-p 0.9, the model samples only from options making up the top 90% of probability mass, ignoring the unlikely tail.
Top-p is an alternative to temperature for controlling randomness, and the two are often available together. Where temperature rescales the whole probability distribution, top-p truncates it — cutting off the long tail of implausible tokens entirely.
The advantage is adaptivity. When the model is confident, the nucleus is small and output stays focused. When many options are genuinely plausible, the nucleus widens and output varies. Temperature applies the same scaling regardless of confidence.
Standard advice is to tune one or the other, not both. Most practitioners adjust temperature and leave top-p near its default of 0.9-1.0.
Related terms
Temperature
Temperature controls how random a model's word choices are. Low values (0-0.3) make output focused and repeatable; high values (0.8-1.2) make it more varied and creative but less reliable.
Determinism
A deterministic system returns identical output for identical input. Language models are only approximately deterministic even at temperature 0, because floating-point and infrastructure variation introduce small differences.
Token
A token is the unit of text an AI model reads and writes — roughly four characters or three-quarters of a word in English. Models price by the token and cap context by the token, so token count determines both what you pay and what fits.
Tokenizer
A tokenizer is the component that splits text into tokens before a model processes it. Different model families use different tokenizers, which is why the same text can have different token counts on GPT versus Claude.
Context window
The context window is the maximum number of tokens a model can consider at once — your prompt, any attached documents, the conversation history, and the response it generates. Exceed it and the earliest content gets dropped.
Prompt
A prompt is the input you give an AI model to produce an output. Effective prompts specify five things: the role the model should adopt, relevant context, the specific task, the output format, and any constraints.
Put this into practice
Understanding the term is step one. Our free courses and tools let you actually use it.