Why this session exists
Session 3 left one dial unexplained: k. Ask k-means for five clusters and it delivers five; ask for twelve and it delivers twelve, equally confident, on the same data. Somebody has to choose - and in most real projects that somebody negotiates with a stakeholder who "feels like" there should be three segments, or twelve, or one per product line. Today you get the evidence tools that turn that negotiation from taste into argument.
Silhouette: grading without an answer key 7 min live
With no labels, a cluster quality score can only ask geometric questions. Silhouette asks the right one, per customer: are you close to your own cluster and far from the next-nearest one? Average that over everyone and you get one number between -1 and 1. It is the only mainstream clustering score that needs no ground truth - which is exactly why it is the one this course leans on.
LiveReading the number like a practitioner4 min▶
Averages over the whole pile, rules of thumb that hold up:
| Average silhouette | Read |
|---|---|
| 0.6 and up | Strong, well-separated structure - our scaled Mango Lane pile at k=5 scores 0.68 |
| 0.4 - 0.6 | Real but soft structure; clusters exist, borders are busy |
| 0.25 - 0.4 | Weak - could be structure, could be noise wearing makeup. Session 9's uniform-noise pile scores 0.31 here |
| Below 0.25 | Stop. You are probably slicing fog |
Two working habits: compare silhouettes across k on the same data (that comparison is today's lab - absolute thresholds are softer than relative ones), and when two k values score close, break the tie with the business, not the third decimal. On our pile k=4 scores 0.66 and k=5 scores 0.68 - the metric alone cannot pick that fight; the actionability test (part 3) can.
Self-studyWhat silhouette cannot see2 min▶
Silhouette rewards compact, round, well-separated blobs - the same geometry k-means likes. On banana-shaped or nested clusters (session 5's moons) it under-scores structure DBSCAN finds easily. And it says nothing about usefulness: five geometrically perfect segments that all get the same campaign are still one segment to marketing. Treat it as a necessary check, never a sufficient one.
The elbow method: named, understood, demoted 6 min live
You will hear "just use the elbow method" in every clustering conversation of your career, so you need to know it - and know why this course does not trust it. The recipe: plot inertia (k-means' own objective, from session 3) against k, and pick the k where the curve stops dropping steeply - the "elbow".
LiveWhy the elbow flunks, in two sentences3 min▶
Sentence one: inertia mathematically always decreases as k grows (more centroids = shorter distances), so the plot always slopes down and flattens - even on uniform random noise, where researchers have shown the curve traces a smooth 1/k shape that human eyes happily read elbows into. Sentence two: "where the bend is" is a judgment call with no definition, so two analysts on the same plot pick different k and both cite "the elbow method" as their evidence.
The criticism is not fringe: a 2023 methods paper carries the title "Stop using the elbow criterion for k-means", and the method's own Wikipedia page calls it subjective and unreliable. Know it because your colleagues will use it; demote it because you know what it does on noise.
Self-studyIf not the elbow, what do the pros plot?2 min▶
Silhouette-versus-k (today's lab plots it for you), and when stakes are high, a stability check: re-run the clustering on random subsamples and measure how often customer pairs stay together. Structure that is really there survives resampling; artifacts dissolve. Session 7 makes stability part of the shipping checklist.
The actionability test 5 min live
Silhouette grades geometry. Nobody ships geometry. Mango Lane's marketing team will fund a welcome flow, a win-back push, a VIP perk - each needs an owner, a budget, and enough customers to move a number. That gives you a second, ruthless filter on k, and it is not negotiable just because it is not mathematical.
LiveThree questions per segment, asked out loud3 min▶
- Is it big enough to fund? A 14-customer segment cannot pay for its own email template. Our lab flags anything under ~6% of the pile as "too small to fund" - your real threshold comes from your campaign economics.
- Would we treat it differently? If segments 3 and 7 both get "the discount email", they are one segment wearing two ids. Merge until every segment implies a distinct action.
- Can someone own it? A segment nobody claims in the campaign calendar is a chart, not a segment.
Notice these questions cap k from above. The data's structure caps it from below (silhouette). The right k lives where both are satisfied - on our pile, that is 5, comfortably. When a stakeholder pushes for more "granularity", you now have two independent walls to point at.
The k lab 🎛 ★ 20 min · everyone turns the dial
The lab below runs REAL k-means (best of five k-means++ restarts) on the scaled Mango Lane pile at every k you ask for, computes the REAL silhouette each time, and shows the segment table the way marketing would read it. The shiny amber button is a trap. Press it anyway - deliberately walking into it here is what keeps you out of it at work.
Run every k. Read the silhouette-by-k curve: it climbs to a peak at k=5 (0.68) and slides off after. That peak is the data's own opinion about how many groups it contains.
Inspect the neighbors. Slide to k=4 (0.66) and k=6, run each. Watch WHAT changes in the segment table: at 4, two behaviorally distinct groups merge; at 6, one real group splits into two rows that would get the same campaign. Close scores, visibly worse tables - this is why ties break on actionability, not decimals.
Press ✨ More segments! (k=12). Feel the pull first - twelve segments genuinely sounds more sophisticated, more "granular", more like a premium consulting deliverable.
Read the wreckage. Silhouette fell off a cliff: 0.68 to 0.34 - noise territory (session 9 will show you actual noise scoring 0.31). The table now has three segments flagged "too small to fund" and several rows whose R/F/M profiles a marketer could not tell apart. Same 300 customers, cut thinner, knowing less.
Note what did NOT fall: the truth match. Still 100% - twelve slices of five real groups still respect the real borders, so purity holds while usefulness dies. A score can stay green while the deliverable rots; this is why no single number gets the final word.
Come home to k=5 and say the sentence out loud: "silhouette peaks at five, every segment is fundable and distinct - the data supports five and the business can staff five." That sentence, with the two walls behind it, is the deliverable of this session.
Homework 📝 ◐ 20 minutes
- Defend a different k: write three sentences arguing for k=4 on our pile (it is defensible!). Feeling how easy the argument is to make is the best vaccine against trusting your own first pick too much.
- Find a segmentation at work and count its segments. Ask (or guess): what evidence chose that number? If the answer is "there have always been four tiers", you have found a k chosen by history - common, and worth knowing.
- Revisit your session-1 hand-sort: how many groups did you make? Given today's tools, would you defend that number with silhouette-style evidence, actionability logic, or admit it was vibes? All three answers are instructive.
Three questions before you go 🎯 ◐ 90 seconds
1 · A customer's silhouette value is near +1 when...
Cohesion small (a), separation big (b), s = (b - a) / max(a, b) approaches 1. Near 0 = on a border; negative = probably filed in the wrong cluster.
2 · The core problem with the elbow method is...
More centroids always means shorter distances - the curve slopes down on anything, including noise, and where it "bends" has no definition. Hence the 2023 paper literally titled "Stop using the elbow criterion for k-means".
3 · At k=12 the lab's truth match stayed 100% while silhouette crashed to 0.34. The lesson is...
Twelve slices of five real groups never mix truths - purity survives - but three segments could not fund a campaign and several were behaviorally identical. Geometry green, business dead: no single metric gets the final word.
What this session covered from the official universe
Certificates, videos, and graded assessments stay with the official providers - this course teaches the working core with a live in-browser lab.