Why knowing the parts changes the conversation
In session a1 you learned the one-breath definition: retrieve the passages that matter, augment the prompt, generate a grounded answer. That is the idea. This session is the machine that makes the idea real. When a vendor or a team says "we built a RAG system", they made nine separate choices - how documents get in, how they were cut up, how they are searched, whether results are re-scored, whether answers are cited, and how anyone knows it works. Most of those choices are invisible until something goes wrong. Learn the nine parts and each one becomes a question you can ask before it goes wrong - which is the whole point of being the person funding the build.
The pipeline on one page 9 min live
Here is the entire machine. Read it left to right in two halves. The first half - documents in, chunked, embedded, stored - is index time: you do it once (and repeat only when documents change). The second half - a question comes in, gets retrieved, reranked, augmented, answered, cited - is query time: it runs on every single question, so it is where you pay per use. Underneath runs an evaluate loop that watches the whole thing.
LiveIndex time vs query time - the split that governs your bill3 min▶
The single most useful line to draw through a RAG system is the line between index time and query time, because they cost money in completely different ways.
- Index time is a one-off (per batch of documents). Ingesting, chunking, embedding, and storing your documents happens once. You pay it again only when documents change or you re-index. It is a project cost, not a running cost.
- Query time is a per-question cost. Every question runs retrieve, rerank, augment, generate, cite. This is the meter that spins all day, forever. A design that is a little wasteful here is a bill that grows with adoption.
Leaders who miss this line fund the wrong thing - they worry about the one-time indexing cost and ignore the per-query cost that actually scales. Session a3 puts real numbers on both. For now, just hold the shape: build once, pay per use.
LiveThe nine parts in plain English4 min▶
Strip away the jargon and every RAG system is these nine jobs. Six are the moving parts leaders fund; the diagram numbers a few sub-steps, but this is the list to memorize.
| Part | What it does, in one line | When |
|---|---|---|
| Ingest | Pull documents in and clean them up | Index |
| Chunk | Cut each document into passage-sized pieces | Index |
| Embed | Turn each passage into a list of numbers that captures its meaning | Index |
| Store | Keep those numbers in a vector database you can search fast | Index |
| Retrieve | For a question, fetch the closest-in-meaning passages | Query |
| Rerank | Re-score the fetched candidates and keep the truly best few | Query |
| Augment | Assemble the passages plus the question into one prompt | Query |
| Generate | Have the model write an answer from the supplied passages | Query |
| Cite | Attach the source each part of the answer came from | Query |
Notice how ordinary each job is. There is no magic - just a pipeline. The skill is knowing which job is weak, which is why Part 2 turns every one of these into a question you can ask.
Self-studyA document becomes "nodes" - the vocabulary you will hear2 min read▶
Your team will not always say "chunk". The most common framework, LlamaIndex, calls the raw input a Document and each chunk a Node - so "we generated 40,000 nodes from 1,200 documents" just means "we cut 1,200 files into 40,000 passages". Do not let the vocabulary hide the plainness of the idea: a node is a chunk, a chunk is a passage, and the size of those passages is a real design choice you will interrogate in Part 2. Knowing the word lets you follow the conversation; knowing it is just a chunk keeps you from being impressed by it.
What each part is for, and the question it earns you 8 min live
The industry has a tidy way to group the nine jobs into five stages, and it is worth borrowing because it is how your team already thinks. LlamaIndex names them Loading, Indexing, Storing, Querying, and Evaluation. The first three are index-time; querying is per-question; evaluation runs across everything. Here they are as a ladder - and then each rung becomes a question you can ask out loud.
LiveEach part, and the question that tells you it was thought through4 min▶
This is the heart of the session. You do not need to build any of these - you need to ask the question that separates a deliberate choice from an accident.
| Part | The question it earns you |
|---|---|
| Ingest | "Which documents are in, which are out, and how do updates get in?" |
| Chunking | "How did we cut our documents, and did we test that choice?" |
| Embedding | "Which embedding model turns our text into meaning, and what did it cost to run over everything?" |
| Store | "Where do the embeddings live, and how fast can we search them?" |
| Retrieval | "How many passages do we fetch, and how do we know they're the right ones?" |
| Reranking | "Do we re-score results before showing them, or do we trust the first search?" |
| Augment | "What exactly ends up in the prompt, and could sensitive text leak in?" |
| Generate | "Does the model answer only from the passages, or is it allowed to improvise?" |
| Citations | "Can every answer point to its source?" |
| Evaluation | "What's our groundedness score, and who owns it?" |
The querying stage, in LlamaIndex terms, is a small chain you will hear named: a retriever fetches candidates, a node postprocessor reranks or filters them, and a response synthesizer writes the answer. Three names, three of your questions - retrieval, reranking, generate.
LiveWhy chunking and citations are the two parts leaders should watch3 min▶
Of the nine parts, two repay a leader's attention far more than the rest.
- Chunking decides what "a passage" even is. Cut too big and the model drowns in irrelevant text; cut too small and a passage loses the context that made it meaningful. This one choice, made early and rarely revisited, quietly caps the quality of everything downstream. Ask whether it was tested, not just picked.
- Citations are your audit trail. A system that cannot point at the source of an answer cannot be trusted in any regulated or high-stakes setting. Citations are not a nice-to-have feature bolted on at the end - they are a design decision made back at chunking, because you can only cite what you tracked from the start.
The assistant that could not cite. A team (anonymized) shipped a helpful internal Q&A bot, then legal asked "show me where each answer came from." The system had chunked documents without keeping track of which file each passage came from - so it could answer but never cite. Retrofitting citations meant re-doing the ingest and chunk steps. A question at design review - "can every answer point to its source?" - would have caught it in an hour instead of a quarter.
Self-studyHow LangChain names the same parts2 min read▶
You will hear the same nine jobs under different names depending on which toolkit your team uses. LangChain, the other common framework, describes its pipeline as: load documents, split them with a text splitter, create embeddings, put them in a vector store, then a retriever runs a similarity search that fetches the top k passages, which are handed to the model to generate. It is the exact same machine - loading, chunking, embedding, storing, retrieving, generating - just with the plumbing named differently. When your team says "we set k to 5", they mean "we fetch the five closest passages", which is your retrieval question in disguise. Do not let two vocabularies convince you there are two different systems; there is one anatomy, and you now know all nine parts of it.
Map your own system's nine parts ★ 12 min · pen and paper
The anatomy becomes yours the moment you draw it for a real system. Take the RAG idea you sorted as a "real RAG project" in session a1 - or one your team already runs - and lay its nine parts on paper.
Write the nine parts down the left of a page: ingest, chunk, embed, store, retrieve, rerank, augment, generate, cite. Draw the index-time / query-time line after "store".
For each part, mark one of three things: exists (we do this on purpose), missing (we do not do this at all), or accidental (it happens but nobody decided how).
Now write the owner's name next to each part. Any part with no name is a risk - it means a real decision is being made by default, not by a person.
Circle the one part that is both important and un-owned. That circle is the single most valuable output of this exercise - it is the conversation to have this week.
The "which part is weakest" walkthrough ★ 8 min · a system you have
A traffic-light pass over the nine parts turns a vague worry - "I'm not sure our bot is any good" - into a specific, fundable fix. Rate each part red, yellow, or green.
Go part by part and assign a colour. Green: we chose this deliberately and tested it. Yellow: we chose it but never checked it works. Red: it is missing, or nobody knows.
Pay special attention to the two you learned to watch: chunking (was the cut tested?) and citations (can every answer point to a source?). A red on either is a headline finding.
Look at evaluation last. If evaluation is red, quietly downgrade your confidence in every green - because a green you never measured is really a yellow.
Pick the single reddest part that is also cheap to fix. That is your first move - and session a3 will help you see why the cheap fixes are usually chunking, reranking, and a refusal threshold, not a bigger model.
Before session a3 ◐ 30 min total
- Finish the nine-part map for at least one real system, with owners named. Keep it - session a3 puts cost on each part, and a4 puts trust on evaluation.
- Run the traffic-light pass and write one sentence naming your reddest, cheapest-to-fix part.
- Ask one team member "who owns evaluation?" and notice whether you get a name and a number, or a shrug.
- Optional: skim the builder track's b2 session on chunking - you do not need the code, but seeing chunks get cut makes the "was it tested?" question concrete.
- Can you walk me through our nine parts - ingest to cite - and tell me which are deliberate and which just happen?
- How did we chunk our documents, and did we test that choice against any alternative?
- At query time, how many passages do we retrieve, and do we rerank them before the model sees them?
- Can every answer point to the exact source passage it came from - and if not, why not?
- What is our groundedness score, who owns evaluation, and how often do they look at it?
Official sources covered
The leader track teaches from published engineering guidance and course curricula - no vendor marketing. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · The difference between index-time and query-time work is best described as...
Index once, then pay per question. Index-time work is a project cost; query-time work is the running cost that grows with adoption.
2 · "Chunking" in a RAG system means...
Chunking cuts documents into passages - LlamaIndex calls each one a Node. The size of the cut is a real, testable design choice that caps downstream quality.
3 · Which part of the anatomy most directly enables an answer to point at its source?
Citations are only possible if the system tracked each passage's source from the start. That is why "can every answer cite its source?" is a design-review question, not a final-feature request.