What Is RAG? (Retrieval Augmented Generation)

Updated July 23, 2026 · Data as of 2026-07-09

Retrieval augmented generation is a pattern where the system searches your documents for passages relevant to a question, puts those passages into the request, and asks the model to answer using them. The model is unchanged; the input is enriched.

The problem RAG solves is that a model only knows what it saw during training, and it never saw your files. The naive fix is to paste everything into the request, which fails as soon as your material is larger than the context window and wastes money long before that. Retrieval fixes it by fetching only the passages that bear on the question, typically a handful of paragraphs, and sending those.

Most implementations work by splitting documents into chunks, converting each chunk into a numeric representation called an embedding, and storing those in a searchable index. A question is converted the same way, and the system returns the chunks that sit closest to it in meaning rather than in exact wording. That is why a good retrieval system finds a passage about billing cycles when you ask about being charged twice, even with no shared keywords.

The quality ceiling is set by retrieval, not by the model. If the right passage is not fetched, a stronger model cannot rescue the answer; it will produce something fluent and unsupported instead. This is why serious systems show their sources. Citations are not decoration, they are the only cheap way for you to see whether the answer rests on the right passage or on nothing at all.

Why it matters when you choose a tool

Any product that promises to answer questions over your own documents, notebooks, wikis, or codebases is doing some form of retrieval, and the differences between them live there rather than in the model they use. Judge these tools on whether they cite sources, whether they cope with tables and images rather than plain prose, and whether they admit when nothing relevant was found. A tool that never says it does not know is not being confident, it is failing quietly.

Frequently asked questions

Is RAG better than fine-tuning?

For teaching a model facts, almost always yes: it is cheaper, updates instantly when a document changes, and shows its sources. Fine-tuning is the better tool for teaching a model a style, a format, or a task pattern, which retrieval cannot do.

Why does the tool cite a document that does not answer my question?

Retrieval found the closest passage available, which is not the same as a correct one. It usually means the source material does not contain the answer, or the question uses vocabulary that does not appear in your documents.

Does RAG stop hallucination?

It reduces it substantially by grounding answers in real passages, and it makes the remaining errors easier to catch because you can check the citation. It does not eliminate the problem, because a model can still misread or overreach on a passage it was given.

Back to the glossary · Take the quiz · How we score tools