Open the editor

Retrieval-augmented generation (RAG)

Find the relevant documents first, then let the LLM answer from them.

Retrieval-augmented generation (RAG) diagram: Documents, Chunk and embed, Embedding model, Vector store, RAG app, LLM

Retrieval-augmented generation answers questions with a large language model, but grounds the answer in your own documents. Before asking the model, the application searches a knowledge base for the passages most relevant to the question and puts them into the prompt. The model answers from that context, and can cite it, instead of relying only on what it learned in training.

A RAG system has two halves. Indexing splits documents into chunks, turns each chunk into an embedding (a vector that captures its meaning) and stores them in a vector store. At question time, the question is embedded the same way and the nearest chunks are retrieved.

The parts of the pattern

Documents
The knowledge the assistant should use: PDFs, wiki pages, tickets, code.
Chunk and embed
Splits documents into passages and calls the embedding model for each one.
Embedding model
Turns text into vectors. The same model is used for chunks and questions.
Vector store
Finds the chunks whose vectors are closest to the question's, often with keyword search alongside (hybrid search).
RAG app
Retrieves, optionally re-ranks, builds the prompt with the chunks and their sources, and checks the answer.
LLM
Generates the answer from the question and the retrieved context.

How it works

  1. Offline, each document is chunked, embedded and stored with a link to its source.
  2. A user asks a question; the app embeds it with the same model.
  3. The vector store returns the top few matching chunks.
  4. The app sends the question and those chunks to the LLM, with an instruction to answer only from them.
  5. The answer comes back with citations to the source documents.

When to use it, and when not to

Good fit

  • Assistants over private or frequently changing knowledge: docs, support, policies, code.
  • Answers must cite sources and stay current without retraining a model.
  • You want access control on knowledge: retrieve only what the user is allowed to see.

Think twice

  • The task needs reasoning over a whole corpus at once (summarise every ticket) rather than finding a few passages.
  • The knowledge is small and stable enough to put directly in the prompt.
  • No plan to measure retrieval quality: most RAG failures are retrieval failures, not model failures.

Edit this diagram

Click Open the diagram in the editor to get your own copy, saved in your browser. Rename the boxes to match your system, swap the generic boxes for technology icons from the palette, then export a PNG, SVG or PDF, or share a link. No sign-up.