From finding to ordering
Session 3 gave us embeddings - the trick of turning words and documents into points in a meaning-space, where closeness means similar meaning. Search is where that trick pays off in public. But finding matches is only half the job: the instant a query returns fifty results, the real question is which one goes first. That ordering is not handed down by nature - it is a weighted blend of signals that a human chose, and changing the weights changes what wins. Get literate about that and you can read any feed, any results page, any "the algorithm changed" headline. Then we point search at your own documents and get RAG, the pattern behind most enterprise GenAI.
From keyword to semantic search 5 min live
Old search matched the letters you typed. If you searched "won't turn on" and the manual said "power failure", keyword search shrugged - no shared words. Semantic search fixes exactly this by comparing meaning using embeddings (Session 3): both phrases land near each other in meaning-space, so the right answer surfaces even with zero words in common.
LiveWhy semantic wins - and where keyword still rules3 min▶
- Keyword search matches literal strings. Fast, exact, and blind to meaning: "car" will not find "automobile", and "won't turn on" will not find "power failure".
- Semantic search embeds the query and the documents into meaning-space (Session 3) and returns the nearest ones. It catches synonyms, paraphrases, and even typos, because it compares meaning, not spelling.
- Semantic wins whenever people ask in their own words - support tickets, natural-language questions, "the thing that does X".
- Keyword still matters when the literal string is the answer: product codes, part numbers, error codes, names, legal citations. You want an exact match, not a close-in-meaning one.
A support team swapped keyword-only search on their help centre for semantic search and watched deflection jump: customers typing "my screen is frozen" finally reached the "unresponsive display" article that shared not one word with their query. The articles did not change - only how meaning was matched.
Self-studyHybrid search: why real systems combine both2 min read▶
In practice almost no serious system is purely one or the other. Hybrid search runs keyword and semantic together and merges the results, then layers filters on top (in stock, this region, published after a date, this permission level). Keyword guarantees the exact-match cases semantic can fumble; semantic rescues the natural-language cases keyword misses; filters enforce the hard constraints that must never be violated regardless of relevance. When someone says "we added AI search", they usually mean they added the semantic lane beside the keyword one - not that they threw keyword away. The literate expectation is a blend, tuned per use case.
Ranking = weighted signals 5 min live
Search finds many matches; ranking decides their order. And here is the idea that reframes every results page and every feed: the order is a weighted sum of signals - relevance, freshness, popularity, personalization, authority - and the weights are choices a human made. There is no objective #1. Change the weights and a different result wins. You will feel this directly in the tuner below.
LiveThe signal mixer, and who sets the weights3 min▶
- Ranking blends signals. Typical ones: relevance (how well it matches), freshness (how recent), popularity (how many liked or clicked it), personalization (how well it fits you), and authority (how trusted the source).
- The order is a weighted sum. Each signal gets a weight; multiply, add, sort. Crank freshness and today's post beats last year's classic; crank popularity and the crowd's favorite jumps to the top.
- There is no objective #1. "Best result" is undefined without a weighting. Someone at the company decided those weights, and that decision has consequences - it is a product and business choice, not a fact about the world.
- "The algorithm changed" almost always means the weights changed. A feed did not gain a soul overnight; a team re-tuned the mix, and what surfaces shifted.
An entire industry - SEO - exists purely because ranking weights have money attached. Businesses reverse-engineer what a search engine appears to reward (fresh content, links from trusted sites, mobile speed) and reshape themselves to score higher. That whole economy is downstream of one truth: the ranking is a set of weights someone chose, and moving up the list is worth real revenue.
Self-studyPersonalization: the signal that splits the results page2 min read▶
Personalization is the ranking signal that quietly makes your results page different from mine for the same query. It folds in your history, location, device, and past clicks - the same behavioral signals we met in Session 4. Used well, it is genuinely helpful: a search for "python" returns snakes for the zoologist and code for the developer. Used carelessly, it recreates the filter-bubble risk from recommendations - the results narrow to confirm what you already engage with, and two people can no longer even see the same "facts" from the same search. The literate view: personalization is a weight like any other, powerful and worth auditing, not a neutral convenience.
RAG: search feeding an LLM 4 min live
Here is where search becomes the backbone of enterprise AI. Retrieval-augmented generation (RAG) is a simple pipeline: search your own documents for the passages relevant to a question, hand those passages to an LLM as context, and let it answer grounded in your data instead of its training memory. It is the dominant pattern for putting GenAI to work - and it makes the point of Sessions 6 and 7 early: the model is only as good as what you retrieve for it.
LiveWhy RAG is the dominant enterprise pattern3 min▶
- The move: retrieve the handful of your documents most relevant to a question, paste them into the model's context, and ask it to answer using them. The model reads your data at answer-time instead of relying on what it happened to memorize.
- Why it wins: it grounds answers in your current, private, authoritative content - policy docs, product manuals, tickets - so the model is far less likely to hallucinate (Session 6) and can cite where it got each fact.
- Why it is everywhere: no retraining, no fine-tuning, update the answer by updating the documents. That low cost and freshness is why RAG, not fine-tuning, is the default first build for enterprise GenAI (Sessions 6-7 go deeper).
- The catch: the answer can only be as good as what you retrieved. Miss the relevant passage and even the best model confidently answers from the wrong context. Search quality caps answer quality.
Self-studyChunking, and why retrieval is mostly data prep2 min read▶
Before documents can be retrieved they are split into chunks - passages small enough to embed and hand to the model, big enough to carry a complete thought. Chunk too large and retrieval returns a wall of mostly-irrelevant text that crowds the context; chunk too small and you sever the sentence that held the answer. Getting this right - plus cleaning, de-duplicating, and keeping documents current - is where most of the real RAG effort goes. It is the same lesson the data literacy course hammered: 60-80% of the work is data preparation, not modeling. A RAG system is a search system, and a search system is only as good as the corpus you feed it. The AI project is, once again, a data project.
The same results, your priorities ★ 10 min · everyone plays
Five results, three signals. Move the weights and watch the order change in real time. This is the whole idea of Part 2 in your hands: there is no objective best, only your weighting.
Crank relevance to the max and everything else to zero. Note the #1. Now do the same for freshness only, then popularity only. Watch #1 change each time - same results, different winner.
Try to bury the exact-match, highly-relevant doc beneath the others by leaning hard on freshness and popularity. It is unsettlingly easy - and that is the point.
Discuss: in a real product, who decides these weights? What does it mean that the "best" result is really someone's priority list encoded as numbers?
Design your search ★ 6 min · everyone designs
Pick a search box in your world - internal docs, a product catalog, a support-ticket archive. Write down what people actually type into it.
Rank the signals by importance for that box: relevance, freshness, popularity, personalization, authority. Which matters most here?
Find the case where freshness should outweigh relevance (a status page, breaking policy) - and a case where chasing freshness would hurt (evergreen how-to docs, legal reference).
Share your weighting. Notice that two sensible people design different rankings for the same box - because ranking is a choice, not a truth.
RAG readiness ★ 6 min · everyone audits
If you built an internal AI assistant tomorrow, what would it read from - and is that content ready?
List the document sources you would want an internal assistant to answer from: wikis, policy docs, product manuals, past tickets, contracts.
Rate each source on how clean and current it is - fresh and well-structured, or stale, duplicated, and scattered across five tools?
Circle the sources that are not ready. Those are the ceiling on your assistant's answers - garbage retrieval, garbage answer.
Land the realization one more time: the AI project is a data project. The model is the easy part; the corpus is the work.
Teams that launch an internal "ask our docs" assistant and are disappointed almost always find the fault in retrieval, not the model: outdated PDFs, three conflicting versions of the same policy, documents nobody had tagged. The ones that succeed spent their first month cleaning and chunking the corpus - proving, again, that search quality is data quality.
This week ◐ 25 min total
- Keyword or semantic? Notice a search this week that clearly understood your meaning (or clearly did not). Which lane failed you, and would the other have saved it?
- Finish your ranking design from Apply-along 2 - write the signal weights for one real search box and the freshness-vs-relevance edge case.
- Do the RAG-readiness audit from Apply-along 3 - list your sources and honestly rate each on clean-and-current.
- Read the self-study cards above: hybrid search, personalization, and chunking.
Three questions before you go 🎯 ◐ 90 seconds
1 · What is the difference between keyword and semantic search?
Semantic compares meaning in embedding-space; keyword compares the literal string. Each wins somewhere, so real systems blend both (hybrid search).
2 · Why is there no objective "#1 result"?
"Best" is undefined without a weighting. Someone chose those weights - which is why "the algorithm changed" usually just means the weights changed.
3 · What does RAG do, and why does it reduce hallucination?
Retrieval-augmented generation grounds the answer in retrieved documents, so the model has the facts in front of it - reducing (not eliminating) hallucination. Answer quality is capped by retrieval quality.
Concepts covered & where they come from
This course teaches the working intuition of each idea and credits the canon. Leaders get judgment; practitioners get the Self-study lane and the names to go deeper.