Core concepts
Context window
Definition
The context window is the maximum number of tokens a model can consider at once — your prompt, any attached documents, the conversation history, and the response it generates. Exceed it and the earliest content gets dropped.
The context window is a hard architectural limit, not a soft preference. It covers input and output together, which is the detail people most often miss: if you fill the entire window with your prompt, the model has no room left to answer.
Windows have grown quickly. Early models handled 2,000-4,000 tokens; current flagships range from 128,000 to over 1,000,000. But a large window is not the same as effective use of it. Models frequently show a "lost in the middle" effect, attending well to the start and end of a long context while glossing over material buried in the centre.
Practical guidance: reserve 2,000-4,000 tokens for the response, put your most important instructions at the very start or very end, and prefer retrieving the relevant 5% of a document over pasting all of it.
Example
A 200,000-token window sounds enormous, but a 150,000-token codebase plus a long conversation history can still overflow it — and quality often degrades well before the hard limit.
Related terms
Token
A token is the unit of text an AI model reads and writes — roughly four characters or three-quarters of a word in English. Models price by the token and cap context by the token, so token count determines both what you pay and what fits.
RAG (Retrieval-Augmented Generation)
RAG retrieves relevant passages from your own documents and inserts them into the prompt before the model answers. It grounds responses in your data, cuts hallucination, and needs no retraining.
Lost in the middle
Lost in the middle is the tendency of models to use information at the start and end of a long context well while under-weighting material in the middle. Position affects whether the model actually uses what you provided.
Chunking
Chunking splits documents into smaller passages before embedding them for retrieval. Chunk size is a key quality lever: too large dilutes relevance, too small loses the context needed to make sense.
Tokenizer
A tokenizer is the component that splits text into tokens before a model processes it. Different model families use different tokenizers, which is why the same text can have different token counts on GPT versus Claude.
Prompt
A prompt is the input you give an AI model to produce an output. Effective prompts specify five things: the role the model should adopt, relevant context, the specific task, the output format, and any constraints.
Put this into practice
Understanding the term is step one. Our free courses and tools let you actually use it.