Builder session 8 · RAG the 1.x way - learn langchain with phoebe

Part 0

Where DataDesk stands

After b7, DataDesk is a multi-tool agent on a graph spine with persistence and approval gates. It can compute anything about your CSVs - but ask it "how do WE define active user?" and it guesses, because your team's definitions live in wiki pages it has never seen. Tonight we fix that with retrieval, built the 1.x way: an offline ingestion pipeline plus one new tool the agent calls when it decides it needs the docs.

Live - presented in session Self-study - read after class ★ Try it now prompt Official docs + Academy covered

★ What you walk out with today A working ingestion pipeline (load → split → embed → Chroma), a search_docs tool wired into DataDesk, a grounding system prompt that makes it cite sources or say "not in the docs", and a demo that proves the uncomfortable truth: retrieval grounds the model in your data, not in reality.

Part 1 · covers the 1.x retrieval docs + the agentic RAG pattern

Retrieval as a tool 8 min live

RAG in 2026 is two phases with a clean seam. Offline, you build a searchable index of your documents once. Online, that index is exposed to the agent as a plain tool - and the agent, not a hardcoded chain, decides whether a question needs it.

🔍 Click to zoom - two phases: offline ingestion, online agentic retrieval

LiveThe 2026 shape: the agent decides to retrieve3 min▶

Pre-1.0 RAG was a pipeline you hardcoded: every question went retrieve → stuff → answer, whether it needed docs or not. That machinery - RetrievalQA, legacy retrievers, the indexing API, the hub - moved to langchain-classic in 1.0. The 1.x pattern is agentic retrieval: wrap your search in a @tool, hand it to create_agent, and let the model decide per question.

"What is 2+2 of our row counts?" - the agent calls csv_stats, never touches the wiki.
"How do we define active user?" - the agent calls search_docs, reads the chunks, cites the page.
"Compare March churn to our churn definition" - it calls BOTH. A fixed chain could never route this; your agent from b3 already can.

Nothing new to learn architecturally: retrieval is tool-calling, which you have been doing since b1. What is new is the offline pipeline that makes the tool worth calling.

LiveThe pipeline that feeds it: load → split → embed → store3 min▶

Embeddings and vector stores stayed first-class in 1.x - only the chain wrappers left. The ingestion pipeline is four honest steps, run offline, once:

Step	What it does	Course pick
Load	Files → Document objects with metadata	Plain file reads for markdown (loaders exist for pdf, html, notion...)
Split	Long docs → overlapping chunks that fit retrieval	`RecursiveCharacterTextSplitter`, 800 chars, 120 overlap
Embed	Each chunk → a vector capturing meaning	`OllamaEmbeddings("nomic-embed-text")` - free, local, both engines
Store	Vectors + text into a searchable index	Chroma, persisted to disk (FAISS is the in-memory alternative)

Local Chroma is plenty for a team wiki. Production-scale picks (pgvector, Pinecone, OpenSearch) are the same interface with different ops - the code you write tonight ports.

Self-studyChunking judgment, the LlamaIndex honesty card, advanced RAG namecheck4 min read▶

Chunking is a judgment call, not a constant. Small chunks (300-500 chars) retrieve precisely but lose context; big chunks (1500+) keep context but blur similarity search and burn tokens. Overlap (10-20%) stops definitions being cut mid-sentence at chunk borders. Markdown with headers? Split on headers first, characters second. The only real answer is the one you will earn in b10: eval it.

The LlamaIndex honesty card. If retrieval quality over messy documents IS your product - contracts, scanned pdfs, thousand-page manuals - LlamaIndex's ingestion and index tuning is deeper than LangChain's. The common 2026 hybrid: LlamaIndex for ingestion, LangGraph for orchestration. Choosing that combination is a design-review answer, not a betrayal of this course.

Namecheck only: Corrective RAG (retry on bad retrieval), Self-RAG (model critiques its own chunks), Adaptive RAG (route by question type). All are LangGraph graphs wrapped around the exact primitives you learn tonight - beyond scope, but no longer beyond you.

★ Try it now (any chat AI)Here are 3 documents my team actually uses: [describe them - format, length, structure]. Recommend a chunk size, overlap, and split strategy for each, and tell me which ONE is most likely to embarrass a naive 800-char splitter, and why.

Part 2 · the discipline that makes RAG trustworthy

Grounding discipline 5 min live

Retrieval without rules just gives the model fancier material to freestyle over. The discipline is a system prompt contract: answer from the retrieved chunks, cite the document, and when the docs are silent - say so instead of guessing.

🔍 Click to zoom - grounded vs ungrounded: the contract is the difference

LiveCite-or-say-so: the three-rule system prompt3 min▶

The grounding contract is three sentences added to DataDesk's system prompt. It costs nothing and changes everything a stakeholder will trust:

★ The grounding rules (paste into your system_prompt)When a question concerns team definitions, metrics, or documentation, call search_docs before answering. Answer ONLY from the retrieved chunks and name the source document for every claim, like: (source: metrics.md). If the retrieved chunks do not contain the answer, reply exactly: "Not in the docs" - then say what you would need. Never fill gaps from general knowledge without labeling it as a guess.

Rule 1 makes retrieval happen for the right questions - the agent decides, but you set the policy.
Rule 2 makes every answer checkable. A citation your analysts can click beats eloquence.
Rule 3 is the one that builds trust: a model that can say "I do not know" is a model people stop double-checking.

Real world

The metric with three definitions. A commerce data team found "active user" defined three ways across three dashboards. Their wiki-grounded assistant, forced to cite, surfaced the conflict in week one - because two answers came back citing different pages. The citation rule did not just prevent hallucination; it exposed a governance bug humans had shipped around for a year.

Self-studyEmbeddings choices + when RAG beats long context3 min read▶

Embeddings model choice. The course uses nomic-embed-text via OllamaEmbeddings: free, local, and it keeps the fully-private path intact - your documents never leave the machine even when the chat model is Claude. Hosted embeddings (Voyage, OpenAI) score higher on retrieval benchmarks; swap the one constructor line if your docs are not sensitive. One hard rule: query and index must use the same embeddings model - change it and you re-ingest.

When RAG beats long context - and vice versa. Models now take 200k+ tokens, so why not paste the whole wiki into every request? For a handful of stable pages, honestly - do that, it is simpler. RAG wins when the corpus is bigger than the window, changes often (re-embed one file vs re-paste everything), needs per-question cost control (3 chunks vs 200 pages of tokens per call), or needs citations with provenance. DataDesk's wiki will outgrow a context window; your five-page runbook may never. Choose per corpus, not per fashion.

Demo 1 of 2

DataDesk reads the team wiki ★ 12 min · everyone builds

Ingest a tiny synthetic wiki into Chroma, wrap similarity search as a tool, hand it to DataDesk, and watch the agent decide to retrieve - then cite.

Seed the wiki: create wiki/ with 3-5 short markdown files - metrics.md (define active user, churn, MAU), data_dictionary.md (columns of your demo CSV), oncall.md. Synthetic content only; write definitions with real opinions in them.

Install the retrieval pieces: pip install langchain-chroma langchain-ollama langchain-text-splitters and ollama pull nomic-embed-text.

Run the ingestion script below once. Peek at ./wiki_db - your wiki is now vectors on disk.

Add search_docs to DataDesk's tool list and the grounding rules to its system prompt. That is the whole integration.

Ask: "How do we define active user?" Watch the stream: the agent chooses search_docs, gets chunks back, answers with (source: metrics.md). Then ask a pure-CSV question and confirm it does NOT retrieve. The routing is the lesson.

★ Ingestion + the retrieval tool - the whole thingfrom pathlib import Path from langchain_core.documents import Document from langchain_text_splitters import RecursiveCharacterTextSplitter from langchain_chroma import Chroma from langchain_ollama import OllamaEmbeddings from langchain.tools import tool emb = OllamaEmbeddings(model="nomic-embed-text") # ---- offline: ingest.py - run once, re-run when the wiki changes ---- docs = [Document(page_content=p.read_text(), metadata={"source": p.name}) for p in Path("wiki").glob("*.md")] chunks = RecursiveCharacterTextSplitter( chunk_size=800, chunk_overlap=120).split_documents(docs) Chroma.from_documents(chunks, emb, persist_directory="./wiki_db") print(f"ingested {len(chunks)} chunks") # ---- online: add to datadesk.py ---- store = Chroma(persist_directory="./wiki_db", embedding_function=emb) @tool def search_docs(query: str) -> str: """Search the team wiki for metric definitions, data docs and runbooks.""" hits = store.similarity_search(query, k=3) return "\n\n".join( f"[source: {h.metadata['source']}]\n{h.page_content}" for h in hits) # then: tools=[csv_stats, column_mean, search_docs] in create_agent, # and the grounding rules appended to system_prompt.

Both engines, one index Embeddings run locally on Ollama regardless of which chat engine answers - so the Claude path and the llama3.1 path share the same wiki_db. Swap the chat model with the usual one line; the index does not care.

Demo 2 of 2

Prove the grounding ★ 8 min · build your own

Two stress tests: one the agent should pass, and one it cannot - because retrieval is grounding, not truth.

Ask something the wiki does not cover: "What is our policy on sharing dashboards with vendors?" A grounded DataDesk retrieves, finds nothing usable, and answers "Not in the docs". If it improvises a plausible policy instead, tighten rule 3 and re-run - watch the refusal appear.

Now poison the well: add metrics_v2.md defining active user as "any user who has EVER logged in" - deliberately wrong. Re-run ingestion.

Ask about active user again. DataDesk may now cite the wrong doc - faithfully, with a tidy (source: metrics_v2.md). It did everything right. The data lied.

Discuss: retrieval grounds the model in your documents, so your documents can now hallucinate FOR it. Garbage in the index is worse than garbage in training data, because it arrives wearing a citation.

Close the loop: the fixes are governance, not prompts - curate what gets ingested, date-stamp docs, prefer one source of truth per definition, and (b10 preview) put "cites the RIGHT doc" into your eval set.

Real world

The stale runbook incident. An ops assistant grounded on a wiki confidently walked an engineer through a decommissioned failover process - citing the page correctly. The page was two years stale. Post-mortem fix: ingestion filter on last-modified date plus a "verified" front-matter flag. Citation discipline caught it fast; index curation stopped it recurring.

Homework

Try it yourself - this week ◐ 30-45 min total

Finish both demos, including the poisoned-doc experiment - feeling the wrong citation land is the point of the session.
Swap 2-3 of your team's REAL (non-confidential) docs into wiki/, re-ingest, and ask five questions your teammates actually ask. Note every miss and every wrong chunk.
Re-run one miss with chunk_size=400 and then 1500. Watch retrieval quality move with a number you chose in one second. Write down which won and your guess why.
Run the chunking-judgment prompt from Part 1 on your gnarliest real document. Bring its verdict to b9.
Optional reading: skim a Corrective RAG or Self-RAG writeup - you now recognize every primitive inside it.

Source material

Official sources covered

This track teaches from the official docs and the free LangChain Academy curricula (login required for lesson content; certificates stay with the Academy - all free). This page covers:

◐LangChain retrieval docs (1.x)Parts 1-2 · the 1.x-native pieces: splitters, embeddings, vector stores, retrieval-as-a-tool

✓langchain-classic migration caveatPart 1 · RetrievalQA, legacy retrievers, indexing API and hub = 0.x history, pip install langchain-classic

✓langchain-chroma + langchain-ollama embeddings docsDemo 1 · Chroma.from_documents, OllamaEmbeddings, similarity_search patterns verified

◐Community RAG curriculum (DeepLearning.AI + tutorial corpus)Framing only · most of that corpus teaches the 0.x chain shape - date-check every tutorial

Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · The 1.x-native RAG shape is...

Agentic retrieval: the fixed chain is 0.x history living in langchain-classic. In 1.x, retrieval is a @tool and the agent routes - which is why DataDesk needed only one new tool, not a new architecture.

2 · A tutorial builds RAG with RetrievalQA and a retriever pipeline from the hub. What do you know?

Same era-reading skill as b1's pipe-operator check. Legacy chains, retrievers, indexing and the hub moved out in 1.0; the primitives you used tonight are what remained first-class.

3 · You seed a deliberately wrong definition doc and DataDesk cites it perfectly. What did the demo prove?

RAG moves the trust problem; it does not solve it. A wrong doc arrives wearing a citation - the fixes are curation, provenance, one source of truth per definition, and evals (b10).

Builder session 8 cheat sheet · pin this

RAG, 1.x shapeRetrieval is a @tool the agent decides to call. Fixed retrieve-then-answer chains = 0.x history in langchain-classic.

Two phasesOffline: load → split → embed → store (run once). Online: agent calls search_docs when needed.

Course pipelineRecursiveCharacterTextSplitter(800/120) → OllamaEmbeddings("nomic-embed-text") → Chroma(persist_directory).

The tool@tool search_docs(query) → similarity_search(query, k=3) → chunks with [source: file] prefixes.

Grounding contractRetrieve for doc questions · answer only from chunks, cite the doc · say "Not in the docs" instead of guessing.

Grounding ≠ truthA wrong doc gets cited faithfully. Curate the index, date-stamp, one source of truth per definition.

One index, both enginesEmbeddings stay local on Ollama; the chat model swaps Claude ↔ llama3.1 without re-ingesting. Never mix embeddings models.

Honesty cardRetrieval-first product? Consider LlamaIndex ingestion + LangGraph orchestration. RAG vs long context: choose per corpus.