Models & architecture
Multimodal model
Definition
A multimodal model accepts or produces more than one type of data — commonly text plus images, and increasingly audio and video. It processes them in a shared representation rather than through separate pipelines.
Practically, multimodality means you can hand a model a screenshot, a chart, a PDF page or a photo and ask questions about it in natural language. This has quietly become one of the most useful capabilities in day-to-day work: debugging from a screenshot, extracting data from a photographed table, describing a diagram.
Capability varies a lot by modality and by task. Reading text in images (OCR-style tasks) is generally strong. Precise spatial reasoning — exact positions, counting many similar objects, reading cluttered charts — remains noticeably weaker than text reasoning.
Note that images consume tokens too, often substantially. A high-resolution image can cost as much as several pages of text.
Related terms
Large language model (LLM)
A large language model is a neural network trained on vast amounts of text to predict the next token in a sequence. That single objective, at sufficient scale, produces the ability to write, summarise, translate, reason and code.
Token
A token is the unit of text an AI model reads and writes — roughly four characters or three-quarters of a word in English. Models price by the token and cap context by the token, so token count determines both what you pay and what fits.
Inference cost
Inference cost is what you pay to run a model on a request, almost always billed per million input and output tokens. Output tokens typically cost three to five times more than input tokens.
Transformer
The transformer is the neural network architecture behind virtually all modern language models. Its key innovation is self-attention, which lets every token directly consider every other token in the input.
Attention
Attention is the mechanism that lets a model weigh how much each token should influence the interpretation of every other token. It is how a model resolves what a pronoun refers to, or which clause modifies which noun.
Parameters
Parameters are the learned numerical weights inside a model, adjusted during training. Parameter count — 7B, 70B, 405B — is a rough proxy for capacity, but not a reliable predictor of quality.
Put this into practice
Understanding the term is step one. Our free courses and tools let you actually use it.