AI Glossary

An AI agent is a system where a model plans and takes actions through tools in a loop, rather than producing a single response. It decides what to do next, does it, observes the result, and continues until the goal is met.

Attention is the mechanism that lets a model weigh how much each token should influence the interpretation of every other token. It is how a model resolves what a pronoun refers to, or which clause modifies which noun.

Bias in AI is systematic skew in outputs that reflects patterns in training data or training process — including demographic stereotypes, cultural assumptions, and over-representation of dominant viewpoints.

Chain-of-thought prompting asks a model to work through its reasoning step by step before answering. It measurably improves accuracy on maths, logic, and multi-step problems by forcing intermediate steps instead of an immediate guess.

Chunking splits documents into smaller passages before embedding them for retrieval. Chunk size is a key quality lever: too large dilutes relevance, too small loses the context needed to make sense.

A citation links a model claim to the source it came from. Real citations come from grounding in retrieved documents; models asked to cite from memory frequently invent plausible-looking references.

A confidence score estimates how reliable an output is. Language models do not natively produce calibrated confidence, and asking a model how sure it is yields poorly calibrated self-assessment.

Context engineering is the practice of deciding what information enters a model's context window and in what order. As windows grew, choosing what to include became more consequential than wording the instruction.

The context window is the maximum number of tokens a model can consider at once — your prompt, any attached documents, the conversation history, and the response it generates. Exceed it and the earliest content gets dropped.

Cosine similarity measures how closely two vectors point in the same direction, on a scale from -1 to 1. It is the standard way to compare embeddings, because it captures semantic similarity while ignoring text length.

A deterministic system returns identical output for identical input. Language models are only approximately deterministic even at temperature 0, because floating-point and infrastructure variation introduce small differences.

Distillation trains a smaller model to imitate a larger one, transferring much of the capability at a fraction of the inference cost. Most small production models are distilled from bigger teachers.

An embedding is a list of numbers representing the meaning of a piece of text, such that semantically similar texts have mathematically similar vectors. Embeddings make meaning-based search possible.

Evaluation is systematically measuring AI output quality against a test set, rather than judging from a handful of examples. It is what makes prompt and model changes safe to ship.

Few-shot prompting means including two to five worked examples of the input-output pattern you want before making your real request. It is the fastest way to lock in a consistent format, tone or edge-case behaviour.

Fine-tuning continues training a pre-trained model on your own examples to specialise its behaviour. It is the right tool for teaching consistent style, format or classification behaviour — not for adding knowledge.

Function calling lets a model request that your code run a specific function with specific arguments. You expose tool definitions; the model returns a structured call, your code executes it, and the result goes back into the conversation.

Grounding means tying model output to verifiable sources — retrieved documents, search results, or database records — rather than relying on parametric memory. Grounded answers can be checked.

Guardrails are the constraints that keep an AI system inside acceptable behaviour — input validation, output filtering, topic restrictions, action limits and human approval gates.

A hallucination is model output that is fluent and confident but factually wrong — invented citations, non-existent functions, fabricated statistics. It stems from models being trained to produce plausible text, not verified truth.

Human in the loop means inserting a person at decision points in an automated workflow — typically to approve consequential actions or review low-confidence outputs before they take effect.

Inference cost is what you pay to run a model on a request, almost always billed per million input and output tokens. Output tokens typically cost three to five times more than input tokens.

JSON Schema is a standard vocabulary for describing the shape of JSON data — its keys, types and required fields. AI providers use it to constrain model output to a guaranteed structure.

A model's knowledge cutoff is the date after which it has no training data. It cannot know about events, releases or prices after that point unless given the information in the prompt or via search.

A large language model is a neural network trained on vast amounts of text to predict the next token in a sequence. That single objective, at sufficient scale, produces the ability to write, summarise, translate, reason and code.

Latency is the delay before and during a response. Two measures matter: time to first token, which governs perceived responsiveness, and tokens per second, which governs how fast the rest arrives.

LLM as judge uses one model to grade another's output against criteria you define. It makes evaluating subjective quality practical at scale, with known biases to control for.

LoRA (Low-Rank Adaptation) is a fine-tuning method that trains small adapter matrices instead of updating all model weights. It cuts the compute and storage cost of fine-tuning by orders of magnitude.

Lost in the middle is the tendency of models to use information at the start and end of a long context well while under-weighting material in the middle. Position affects whether the model actually uses what you provided.

MCP is an open standard for connecting AI models to external tools and data sources. It replaces bespoke per-integration code with one protocol any compatible client can use.

Meta-prompting is using a model to write, critique or improve prompts. You describe the task and the failure modes you want to avoid, and the model drafts the prompt.

Mixture of experts is an architecture where only a subset of the model activates per token. A router selects a few specialised sub-networks, so a very large model runs at the cost of a much smaller one.

Model drift is when a hosted model's behaviour changes over time as the provider updates it, causing prompts that worked reliably to degrade without any change on your side.

A multimodal model accepts or produces more than one type of data — commonly text plus images, and increasingly audio and video. It processes them in a shared representation rather than through separate pipelines.

An open-weights model has publicly downloadable parameters, letting anyone run it on their own hardware. Llama, Mistral, Qwen and DeepSeek are prominent examples.

Parameters are the learned numerical weights inside a model, adjusted during training. Parameter count — 7B, 70B, 405B — is a rough proxy for capacity, but not a reliable predictor of quality.

A prompt is the input you give an AI model to produce an output. Effective prompts specify five things: the role the model should adopt, relevant context, the specific task, the output format, and any constraints.

Prompt caching stores the processed form of a repeated prompt prefix so subsequent requests reusing it are cheaper and faster. Cached input tokens typically cost a fraction of normal input tokens.

Prompt chaining splits a complex task into a sequence of focused prompts, where each step's output feeds the next. Chains are easier to debug and more reliable than one large prompt.

Prompt engineering is the practice of designing model inputs that reliably produce the output you want. It combines clear instruction-writing, structured formatting, worked examples, and systematic testing.

Prompt injection is an attack where malicious instructions hidden in content the model processes override the developer's intended behaviour. It is the most serious unsolved security issue in LLM applications.

Quantization reduces the numerical precision of model weights — from 16-bit to 8-bit or 4-bit — shrinking memory use and speeding inference at a modest cost in quality.

RAG retrieves relevant passages from your own documents and inserts them into the prompt before the model answers. It grounds responses in your data, cuts hallucination, and needs no retraining.

ReAct (Reason + Act) is the core agent loop: the model reasons about what to do, takes an action via a tool, observes the result, then reasons again. It repeats until it can answer.

A reasoning model is trained to generate extended internal deliberation before answering. It trades latency and cost for substantially better performance on maths, logic, coding and multi-step problems.

Reranking takes an initial set of retrieved candidates and reorders them with a more accurate but slower model. It is one of the cheapest ways to materially improve RAG quality.

Self-consistency runs the same reasoning prompt several times at non-zero temperature and takes the most common answer. It trades cost for accuracy on problems with a single verifiable result.

Semantic search finds results by meaning rather than exact keywords, using embeddings to compare concepts. It matches "how do I get my money back" to a document titled "Refund Policy".

Streaming sends a response token by token as it is generated rather than waiting for completion. It does not reduce total generation time but dramatically improves how fast the response feels.

Structured output means constraining a model to return data in a machine-readable shape such as JSON, usually by specifying the exact schema in the prompt or using a provider feature that enforces it.

A system prompt is a standing instruction that shapes a model's behaviour across an entire conversation, separate from individual user messages. It typically defines persona, rules, tone and output format.

Temperature controls how random a model's word choices are. Low values (0-0.3) make output focused and repeatable; high values (0.8-1.2) make it more varied and creative but less reliable.

A token is the unit of text an AI model reads and writes — roughly four characters or three-quarters of a word in English. Models price by the token and cap context by the token, so token count determines both what you pay and what fits.

A tokenizer is the component that splits text into tokens before a model processes it. Different model families use different tokenizers, which is why the same text can have different token counts on GPT versus Claude.

Tool use is a model invoking external capabilities — search, code execution, database queries, APIs — to do things it cannot do from parameters alone, such as fetching current data or performing exact arithmetic.

Top-p limits the model to choosing from the smallest set of tokens whose combined probability exceeds p. At top-p 0.9, the model samples only from options making up the top 90% of probability mass, ignoring the unlikely tail.

The transformer is the neural network architecture behind virtually all modern language models. Its key innovation is self-attention, which lets every token directly consider every other token in the input.

Tree-of-thought extends chain-of-thought by exploring several reasoning branches in parallel, evaluating each, and pursuing the most promising. It helps on problems where the first plausible path is often wrong.

A vector database stores embeddings and finds the most similar ones to a query vector quickly. It is the retrieval layer of most RAG systems. Examples include Pinecone, Weaviate, Qdrant and pgvector.

Zero-shot prompting means asking a model to do a task with no worked examples — just an instruction. It works well for common tasks the model has seen extensively in training.