AI agents and technology
What is RAG? Definition and examples
Also: Retrieval-augmented generation, semantic search
In short
RAG (retrieval-augmented generation) is a method where an AI model looks up relevant passages in your documents before it answers. It then answers with your knowledge, not only with its training. The answer is only as good as the sources it finds.
Why it matters: RAG brings your own knowledge into AI answers without retraining a model.
- 01
Question
A person or an agent asks
- 02
Retrieve
Relevant passages in your sources
- 03
Augment
The passages go to the model
- 04
Generate
Answer with source and date
Why does RAG matter for your company?
A language model does not know your prices, customers or decisions. RAG solves this without training: for each question a system finds the relevant sections, usually through semantic search, and hands them to the model. The model writes an answer from them, ideally with sources.
The limit lies in the sources. RAG finds what sounds similar, not what applies: if an old and a new strategy paper sit side by side, the old one can win. RAG therefore needs maintained sources with a clear state, or AI repeats outdated assumptions, even with a citation.
RAG in detail
Semantic search
Semantic search finds content by its meaning instead of exact words. Searching for “notice period” also finds “contract ends with three months' notice”. It is the basis of RAG and of many AI assistants in companies.
Classic search compares words. Semantic search turns text into rows of numbers called embeddings and compares their meaning. It finds answers across languages, synonyms and tools.
Example: The assistant cites the old strategy
A company builds an internal assistant with RAG over SharePoint and Confluence. An employee asks which markets have priority in 2027. The assistant answers “USA” and neatly links last year's strategy paper, although “Strategy 2027: focus on the DACH mid-market, no US expansion” now applies.
The team marks the old paper as superseded and points it to the new decision. The assistant reads valid decisions first and gives source and date with every answer. Now the employee finds the right state and sees since when it applies.
3 common mistakes to avoid
Indexing everything
More documents do not mean better answers. Drafts, copies and outdated material end up in the answer too.
Hiding the sources
Without sources nobody can check where an answer comes from. Show source and date with every answer.
Mistaking RAG for memory
RAG reads afresh for each question and remembers nothing. What was decided therefore has to be in the sources.
Step by step
- 1
Collect the questions
Which questions should the assistant answer? The sources you need follow from that.
- 2
Select and clean the sources
Include only maintained sources and mark what has been superseded.
- 3
Show source and date
Every answer points to the passages it comes from.
- 4
Test with real questions
Check regularly whether the answers match the current state.
Bring one important decision.
In 30 minutes with Marc or Simon we show where your work drifts from the decision today.
Meet the Company BrainRead on in the PM Lab
Articles that go deeper.
Related terms
- AI memory →AI memory is the part of an AI system that keeps knowledge beyond a single conversation.
- Hallucination →A hallucination is a statement by an AI that sounds convincing but is false or unsupported.
- Knowledge Graph →A knowledge graph stores knowledge as a network: things such as customers, projects and decisions are nodes, and their relationships are edges.