What are training, retrieval and RAG?

What you will learn

By the end, you will understand:

Separate training, fine-tuning, memory and retrieval
Explain a basic RAG pipeline and its failure points
Design source permissions and evaluation for grounded answers

Visual explainer

See the idea clearly.

Approved documents are parsed, chunked and indexed; a question retrieves candidate passages; the model receives selected context, generates an answer with citations, and a person or evaluator checks it.

Four mechanisms are often confused

Mechanism	Meaning
Pretraining/training	Adjusts model parameters from large datasets; knowledge is represented as learned patterns, not a browsable source library.
Fine-tuning	Further adjusts behavior from examples; useful for style/task patterns, not a simple substitute for current factual retrieval.
Retrieval	Searches an external collection at request time and returns matching material.
Application memory/state	Stores selected prior facts or conversation outside/inside context according to product design.

A basic RAG path

01
Approve source collection
02
Extract and clean
03
Split into useful chunks
04
Create searchable representation/index
05
Receive question
06
Retrieve candidate passages
07
Apply permissions/rerank
08
Send context to model
09
Generate answer/citations
10
Evaluate and log

Retrieval can fail before generation

Failure	Effect
Missing document	The answer cannot use a source that was never indexed.
Bad parsing/chunking	Tables, headings or conditions become separated or corrupted.
Weak query match	Relevant evidence is not retrieved.
Stale source	A fluent answer reflects an obsolete version.
Permission leak	A user receives content they were not authorized to access.
Malicious content	Retrieved instructions attempt prompt injection or unsafe tool use.

The model can still misuse good context

It may ignore a passage, combine incompatible policies, over-generalize, misquote or add unsupported knowledge. Require answers to identify sources and allow an explicit “not found in approved sources” outcome.

A citation should point to the exact document/version and support the adjacent claim. Do not cite a whole repository vaguely.

Freshness and ownership

Every source needs an owner, approval status, effective date, expiry/review date and audience permissions. Remove superseded documents from the active index or clearly rank current versions above archives.

A RAG system is not a document-governance replacement. If the source library is contradictory, the answer will inherit the conflict.

Test retrieval separately from answers

Representative questions
Expected source/passages
No-answer cases
Ambiguous wording
Permission boundaries
Old/new version conflict
Tables and scanned PDFs
Multiple languages
Prompt-injection content
Citation correctness
Latency/cost
Human escalation

Choose the simplest knowledge approach

Static rule/FAQ

Best when answers are few, stable and need deterministic control.

Search

Best when users can inspect source documents directly.

RAG assistant

Useful for natural-language synthesis over an approved changing collection, with evaluation and controls.

Fine-tuning

Useful for consistent behavior/format in some tasks; it does not keep a policy library current by itself.

Real-world example

Example: an HR policy assistant with version control

Example

The assistant retrieves only current policies employees are permitted to see. Answers cite the exact section and say when the source does not cover a case. Old policies remain in an archive unavailable to normal retrieval; sensitive investigations are excluded entirely.

Try this

Design one retrieval test set

Choose ten real questions: six answerable, two ambiguous and two absent from the approved documents. Record the expected source, permitted audience and correct no-answer behavior before testing a model.

Common questions

Questions beginners ask.

What is RAG?

Retrieval-augmented generation: retrieving external information at answer time and supplying it as model context.

Is RAG the same as training?

No. Retrieval supplies documents at request time; training changes model parameters.

Does RAG guarantee factual answers?

No. Retrieval, source quality, permissions and model use of context can all fail.

What is chunking?

Splitting source material into searchable units while trying to preserve enough context.

What is an embedding?

A numeric representation used to compare semantic similarity, often as one part of retrieval.

Can RAG search private files safely?

Only when identity, authorization, source permissions, logging and leakage tests are designed correctly.

Should old policies remain searchable?

Only with deliberate archival/version controls; current answers should not silently mix superseded policy.

When is a simple FAQ better?

When the approved answers are few, stable and deterministic control is more valuable than open synthesis.

Assessment

Check what you understood.

5 questions · instant explanations

1. What does retrieval do?

2. Which is a retrieval failure?

3. Why test permissions?

4. What is fine-tuning best understood as here?

5. True or false: a retrieved citation always supports the generated sentence.

Sources