The engine under the hype
Session 1 named the dolls. Deep learning is the one everyone means when they say "AI" today - it powers vision, speech, translation, and the language models in Session 6. But the word "deep" hides something almost embarrassingly simple: a tiny unit that adds and compares, repeated at enormous scale. Once you see the neuron, the whole field stops being magic and starts being engineering - engineering with one honest catch, which is that nobody can fully read what the finished machine learned.
What a neuron actually does 5 min live
Forget biology for a moment. An artificial neuron is a small piece of arithmetic: it takes some inputs, multiplies each by a weight (how much that input matters), adds them up with a bias, and if the total clears a threshold it fires. One neuron draws exactly one straight line - a decision boundary that splits the world in two. The whole miracle of deep learning is what happens when you stack these lines into layers until they bend into any shape you need.
LiveInputs, weights, threshold - and how it learns3 min▶
- Inputs are the things the neuron looks at - pixels, numbers, signals from the layer before it.
- Weights say how much each input matters. A big weight means "pay attention to this one"; a weight near zero means "ignore it". The bias shifts the whole threshold up or down.
- The line. With those weights, one neuron splits its inputs into two sides - a decision boundary. That is all a single neuron can do: draw one straight cut.
- Learning is nothing more than adjusting those weights. Nobody sets them by hand. The network is shown examples, sees how wrong it is, and nudges every weight a little toward "less wrong" - over and over (Part 3).
A spam filter's first layer might have neurons that weight words: "free" gets a big positive weight, your CEO's name a big negative one. No engineer wrote those numbers - the model learned them from millions of emails labeled spam or not. When spammers change tactics, you retrain and the weights shift. The logic lives in the weights, not in code you can read.
Self-studyActivation functions: why the line gets to bend2 min read▶
If a neuron only ever added and compared, stacking layers would still just give you one big straight line - lots of lines added together are still a line. The fix is a small nonlinear step called an activation function (ReLU is the common one: "if the sum is negative, output zero, otherwise pass it through"). That one kink is what lets each layer bend the boundary a little, so many layers together can trace any curve - a circle, a spiral, a face. No nonlinearity, no depth advantage; it is the quiet reason "deep" is worth the trouble.
Why "deep" and why 2012 5 min live
"Deep" is not a compliment - it is a count. It means many hidden layers between input and output. Why bother with many? Because each layer learns higher-level features built from the one before it: raw pixels become edges, edges become shapes, shapes become parts, parts become a face. The network builds its own ladder of understanding, and nobody tells it what rung to put where.
LiveThe 2012 moment - old ideas, new fuel3 min▶
- The ideas were from the 1980s. Neural nets and the training math had existed for decades but were slow, shallow, and easy to beat. The field had mostly moved on.
- 2012 changed the fuel, not the idea. A deep net called AlexNet won the ImageNet image-recognition contest by a stunning margin. What was new: a huge labeled dataset (ImageNet's millions of images), and GPUs - gaming chips repurposed to do the arithmetic thousands of times faster.
- CNNs for images. The winning design was a convolutional neural network - it slides small filters across an image looking for the same feature anywhere it appears (an edge in the corner is still an edge). You do not need the math; you need to know CNN means "the vision workhorse".
Before 2012, image-recognition error rates crept down a percent or two a year. AlexNet dropped the ImageNet error rate off a cliff in a single contest, and within about two years every serious vision team had pivoted to deep learning. It is one of the clearest "the field changed overnight" moments in tech - and a callback to Session 1: the leap was scale (data + GPUs), not a new theory.
Training, and the black-box tradeoff 5 min live
Here is the whole training loop in one breath: show the network an example, measure how wrong its answer was, nudge every weight a tiny bit in the direction that would have been less wrong, and repeat - millions of times. That "nudge toward less wrong" is gradient descent, and the honest picture is a marble rolling downhill on a landscape of error until it settles in a valley. Nobody hand-tunes the weights; the downhill roll finds them.
LiveRolling downhill, and what it costs you3 min▶
- Training = examples + error + nudge, repeated. The network guesses, compares to the right answer, and adjusts weights to shrink the gap. Gradient descent is just the rule for which way is "downhill" toward less error.
- More layers and more data buy more capability - the model can capture subtler patterns. This is why "just add more data" is a real strategy (Session 1).
- But capability trades against explainability. A model with millions of weights has no readable rulebook. It works, and it cannot fully tell you why. This is the black box.
- When the black box is fine: movie recommendations, photo tagging, draft suggestions - low stakes, easy to check, cheap to be wrong. When it is not: loan denials, medical flags, hiring - where a person deserves a reason and the law may demand one. Session 8 turns this into governance.
Self-studyOverfitting and train/test, in the deep-learning context2 min read▶
Callback to data literacy Session 6: a model that memorizes its training examples instead of learning the general pattern is overfit - it aces the practice questions and fails the real exam. Deep nets, with their millions of weights, are especially good at memorizing, so the discipline matters more here, not less. The guardrail is the same: hold back a test set the model never trains on, and judge it only on that. If it scores far better on data it has seen than on data it has not, it memorized rather than learned. When a vendor quotes accuracy, the literate question is "on held-out data the model never saw?".
Be the neuron ★ 10 min · everyone tunes
You are about to do by hand what training automates. Two groups of dots; one straight line; three dials. Move the weights and the bias until the violet line perfectly separates the two groups.
Aim for 100%. Drag w1, w2 and the bias and watch the line swing and slide with each change.
Any dot with a red ring is misclassified - on the wrong side of your line. Your job is to leave zero red rings.
Notice what you are doing: you are hand-searching for weights that separate the classes. That is exactly the job training does - it just does it automatically by measuring error and nudging.
Now imagine this with three dials replaced by millions, across many layers, and no human touching them. The network finds them by rolling downhill toward less error. You just felt, by hand, what a GPU does a trillion times.
Sketch the feature ladder ★ 6 min · everyone sketches
Pick one task as a table: spot a cat in a photo, read the total off a receipt, or detect a fraudulent transaction.
Sketch the ladder a deep net might climb. Low level: what raw signals go in? Mid level: what small patterns build from those? High level: what does the final answer look like?
For the receipt: pixels to edges to digits to "the number after the word total". For fraud: raw fields to "unusual amount" and "new location" to "risky combination".
Now try to name every mid-level feature precisely. You cannot always - the model may invent features with no human name. That gap, right there, is the black box you will govern in Session 8.
Black box: OK or not OK? ★ 6 min · room votes
Four real use cases. For each, the room votes: is an unexplainable "black box" answer acceptable here, or does someone deserve a reason?
Movie recommendation. Vote. Most rooms say black box is fine - low stakes, easy to ignore a bad pick.
Resume screening. Vote. Now it gets uncomfortable - a rejected candidate and an employment regulator may both want the reason.
Tumor flag on a scan. Vote. A doctor needs to know what the model saw to trust or overrule it; a pure black box is hard to sign off on.
Ad targeting. Vote, then name what forces explainability in each case - regulation (lending, hiring, health), ethics, or a customer who will simply ask "why?". Carry this into Session 8.
Lending regulators in many countries require a lender to give the specific reasons for a denial. That single rule quietly rules out the most opaque deep models for credit decisions, or forces a reason-producing layer on top. The technology is legal; the un-explainability is what the law will not accept.
This week ◐ 25 min total
- Beat the neuron: reopen Apply-along 1 and get a clean 100% from a random start. Notice how much the bias alone changes the line - that is the "shift the threshold" move.
- Ladder one real system from your work: name a low, mid, and high level feature it would need, and circle any you cannot name.
- Read the self-study cards above: activation functions (why layers bend the line), and overfitting / train-test in deep learning.
- Optional deep end: look up the 2012 ImageNet result and the error-rate chart. Five minutes that explain why the whole field pivoted.
Three questions before you go 🎯 ◐ 90 seconds
1 · What does a single artificial neuron actually compute?
Inputs times weights, add a bias, compare to a threshold. One neuron draws one straight decision line - simple arithmetic, not biology.
2 · In "deep learning", what does "deep" mean?
"Deep" is just a count of layers. Pixels to edges to parts to object - each layer builds on the last. More layers, more capability.
3 · What is the core tradeoff of adding more layers and data to a deep net?
Millions of weights have no readable rulebook. Fine for movie picks; not fine for loan denials, where a person deserves a reason. Match the box to the stakes.
Concepts covered & where they come from
Working intuition of each idea, credited to the canon. Leaders get the judgment; practitioners get the Self-study lane and the names to go deeper.