Retrieval & data
Vector database
Definition
A vector database stores embeddings and finds the most similar ones to a query vector quickly. It is the retrieval layer of most RAG systems. Examples include Pinecone, Weaviate, Qdrant and pgvector.
Comparing a query against millions of vectors by brute force is too slow for interactive use. Vector databases use approximate nearest neighbour indexes (HNSW, IVF) that trade a small amount of recall for enormous speed gains.
Most also support metadata filtering, so you can restrict a semantic search to a date range, document type or access level. In production this is essential — it is how you enforce that a user only retrieves documents they are permitted to see.
For smaller corpora you may not need a dedicated system. pgvector inside an existing Postgres database handles hundreds of thousands of vectors comfortably and avoids operating another service.
Related terms
Embedding
An embedding is a list of numbers representing the meaning of a piece of text, such that semantically similar texts have mathematically similar vectors. Embeddings make meaning-based search possible.
RAG (Retrieval-Augmented Generation)
RAG retrieves relevant passages from your own documents and inserts them into the prompt before the model answers. It grounds responses in your data, cuts hallucination, and needs no retraining.
Semantic search
Semantic search finds results by meaning rather than exact keywords, using embeddings to compare concepts. It matches "how do I get my money back" to a document titled "Refund Policy".
Chunking
Chunking splits documents into smaller passages before embedding them for retrieval. Chunk size is a key quality lever: too large dilutes relevance, too small loses the context needed to make sense.
Reranking
Reranking takes an initial set of retrieved candidates and reorders them with a more accurate but slower model. It is one of the cheapest ways to materially improve RAG quality.
Knowledge cutoff
A model's knowledge cutoff is the date after which it has no training data. It cannot know about events, releases or prices after that point unless given the information in the prompt or via search.
Put this into practice
Understanding the term is step one. Our free courses and tools let you actually use it.