learn-economics-with-phoebe / Deep-dive session 10 of 10
Learn Economics with Phoebe · Deep-dive session 10 of 10

The elasticity bench, and defending a number

Nine sessions have built one claim: on 203 real products, the price elasticity is about -1.8 when each product is compared with itself, between -1.5 and -2.2 depending on choices that are yours to make, and it is neither the -0.1 a first analysis reports nor the zero that comparing across products produces. Tonight the claim sits on a bench. Every slope is refitted in your browser from the 2,216 rows the moment you touch a control, so the ladder is not a table to believe but a machine to work. There is an anti-lever, the choice everyone reaches for, and it gets the sign of the revenue decision wrong. There is a break button that shuffles prices inside every product and watches the signal fall to zero, because a number you cannot make disappear is a number you do not understand. And there is the sentence you say in the meeting, which is shorter than any of this.

🔴 Bench night Analysts, pricing and growth teams, and the leaders they report to Browser only, no code needed 45 min
0-3 · Welcome 3-14 · Reading the ladder 14-40 · The bench, then the sentence 40-45 · Q&A
Part 0

Where we start

A bench is not a dashboard. A dashboard shows you a number; a bench lets you break it. Everything on this page is recomputed from the same 2,216 real product-month rows that deep-dive session 1 built and session 2 fitted, and the point of doing it live is that you can move the four filters, pick any rung, shift every price by any amount, and watch which numbers move and which do not. A slope that survives the filters and dies under the shuffle is a slope you can defend.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Bench = every number computed in your browser

★ What you walk out with tonight. A ladder you have worked with your own hands, the anti-lever you can now name when somebody proposes it, the placebo that tells you how far a slope is from noise, and one sentence for a leader that carries the range, the mechanism and the caveat without a single decimal that is not earned.

Part 1 · covers the five rungs and what each removes

Reading the ladder

8 min live

Five ways to fit a line through the same rows. Each rung removes one thing the rung below could not, and the number changes by a factor of sixteen on the way up. The bench below has the four fits as buttons and the fifth as a break button, because the fifth is not a fit, it is a test.

FIVE RUNGS, ONE SET OF ROWS the slope on each rung is what the bench computes when you press that button 1 · across products +0.008 2 · pooled rows -0.108 3 · each product with itself -1.788 4 · product and month -1.854 5 · prices shuffled within product about 0 WHAT EACH RUNG REMOVES 1 · nothing: cheap products are simply not popular products 2 · nothing, but rows not products, so a little within-product signal leaks in 3 · "this product is cheap and popular": each product's own mean, subtracted 4 · "the marketplace grew this month": each month's mean, subtracted too 5 · the cause: with prices shuffled, a real signal must vanish, and it does Rung 3 is the answer and rung 5 is the reason to believe it. Rungs 1 and 2 are the numbers a first analysis reports.
🔍 Click to zoom - the factor of sixteen between rung 2 and rung 3 is the whole difference between comparing products with each other and with themselves
LiveHow to read a rung, and the one question to ask of any elasticity5 min▶

Every elasticity anyone reports was fitted on some rows in some way, and the question that sorts the useful ones from the rest is: what was compared with what? Across products (rung 1) compares a bath towel with a phone case, and the slope of +0.008 says price tells you nothing about that comparison, which is true and useless. Pooled (rung 2) throws all product-months in one cloud, and the -0.108 is mostly rung 1 with a little within-product signal leaking through. Within product (rung 3) subtracts each product's own mean price and mean units before fitting, so the only variation left is a product moving against itself. That is the comparison a pricing decision is about, and it reads -1.788.

Rung 4 subtracts each month's mean too, which takes out the marketplace growing, and moves the slope to -1.854. That it moves so little is itself a finding: the growth every product shared was already mostly absorbed by comparing products with themselves. Rung 5 is not a fit. It shuffles the prices inside every product, so that each product keeps its own prices and its own units but the pairing between them is destroyed, and refits rung 3. Two hundred times over, the slope averages -0.002 with a standard deviation of 0.159. The real slope is more than eleven of those standard deviations away.

The standard errors on the bench are the naive kind, computed as if every row were independent, and the page says so. They are a floor on the uncertainty, not a ceiling. The placebo is the honest measure of how far the answer sits from noise, and it is why the break button exists.

Self-studyThe four filters, and what each one is a test of3 min▶

Weight months by units makes a month with 121 sales count for more than a month with 2, and moves rung 3 to -2.216. Price includes freight swaps the seller's price for the buyer's cost, and moves it to -1.881. Drop single-unit months removes the 16.5 percent of rows where one sale happened, and moves it to -1.480. Category restricts the panel to one shelf, where the within-product slope runs from -1.283 (garden tools, 13 products) to -3.883 (bed, bath and table, 22 products) with standard errors wide enough that most of the categories cannot be told apart.

None of the four is the right setting. Together they are the range: -1.5 to -2.2, about -1.8 at the default. A number that stays in that band whatever you press is a number with a shape you can describe, and describing the shape is what defending it means.

Build-along · the bench

Five rungs, computed in front of you

★ 20 min · the main event

Work the rungs in order, one to four. For each, read the verdict line, the six readouts, and the cloud the rung fitted. Then set the price slider to +10 percent and read the revenue line on rung 2 and on rung 3. Then press the anti-lever, and then press the break button on rung 3. Say out loud what changed each time before moving on.

Read it in this order. Slope, then the placebo standard deviation, then the two modelled lines. The first two are computed from the rows and carry the "computed" tag; the last two are arithmetic under a constant-elasticity assumption and carry "modelled", because a 10 percent rise has never been run on these products and the line is a projection, not a measurement.

THE ANTI-LEVER GETS THE SIGN WRONG every price up 10 percent · projected revenue under two slopes · constant elasticity, modelled market-wide slope -0.108 +8.9% units -1.0% within-product slope -1.788 -7.2% units -15.7% 0 THE LEVER EVERYONE PULLS "Use the overall number, it has more data behind it." It does, and the data is about the wrong comparison. Same rows. Opposite sign. A first analysis reports the +8.9. The bench shows it next to the -7.2 so that the choice of comparison is visible as a choice.
🔍 Click to zoom - the red bar is what the anti-lever projects; on the bench, press it and read the revenue line, then press rung 3
LiveWhat the break button proves, and what it cannot4 min▶

Press it on rung 3. Inside every product the prices are shuffled among that product's own months, so the set of prices and the set of units are unchanged and only their pairing is destroyed. The slope collapses to somewhere within about 0.16 of zero. Press it again and the real pairing returns with the real slope. A detector that cannot be made to fail has not been tested; this one fails on demand, in the direction it should.

What it cannot prove is causation. The shuffle shows that the pairing of price and units inside a product carries the slope. It does not show that the price caused the units rather than the seller reading the units and setting the price. Product effects remove "cheap and popular", month effects remove "the marketplace grew", the placebo removes "chance", and none of the three removes "the seller chose". That last step needs a price that was changed for a reason unrelated to demand, which is what an experiment is, and the experimentation course owns it.

Part 2 · covers the sentence for the meeting

Defending the number

6 min live

A number is defended by its shape, not its decimals. The shape of this one is: a range, a mechanism, and a caveat. The range is -1.5 to -2.2 with -1.8 at the default. The mechanism is "each product compared with itself, on 203 products that repriced". The caveat is "an association with a placebo behind it, not an experiment". Anyone who asks for a single decimal is asking you to remove the shape, and the correct answer is the shape again, slower.

WHERE THE REAL SLOPE SITS AGAINST 200 SHUFFLES placebo: prices shuffled within product, refit, 200 times · mean -0.002, sd 0.159 rung 3: -1.788 more than 11 placebo sd away -1.8 -0.9 0 +0.4 the widest shuffle reached 0.444 "Between -1.5 and -2.2, about -1.8, each product against itself on 203 that repriced; an association with a placebo behind it, not a test." That is the sentence. It has a range, a mechanism and a caveat, and nobody can take a decimal off it.
🔍 Click to zoom - the hump is what chance produces on these rows; the amber dot is what the rows produce
LiveThe sentence, and the three questions it survives4 min▶

"So what is the number?" Between -1.5 and -2.2, about -1.8 at the default. A 10 percent rise across the board projects to lose 7 percent of revenue, not gain 9. The range is not hedging; it is the four filters on the bench, and each end of it is a setting somebody could reasonably choose.

"How do you know it is real?" Because it disappears when it should. Shuffle prices inside every product and the slope falls to zero within 0.16. The real slope is more than eleven of those standard deviations away. And because it holds when the rows are cut four ways.

"So we should raise prices, or not?" The number says a uniform rise loses revenue on these products. It does not say why buyers respond, it does not say which product to move, and it does not rule out that sellers were reading demand and setting price after it. The next step is a price test on a few products, which the experimentation course designs, not a spreadsheet extrapolation from this slope.

Self-studyWhat the bench does not show, so you do not claim it2 min▶

It does not show an elasticity for the marketplace: the 203 are the products that sold a lot and repriced, and the honest scope is "for the products that can be read". It does not show a per-product elasticity, because deep-dive session 1 found that a third of those are noise. It does not show a causal effect, for the reason in the break button card. And it does not show what happens at prices outside the range the sellers actually tried, which within a product was a median max-to-min ratio of 1.29; a 30 percent rise is a projection past the data.

I can say which rows the number was fitted on: which orders, which price column, which grain, and the three sieves with their counts. Deep-dive session 1. On Olist: delivered before 1 September 2018, the seller's item price, product by month, 30 items and 3 prices and 6 months, 32,216 to 203.
I can say what was compared with what, and I have the across-products and pooled numbers next to the within-product one so the comparison is visible as a choice. Deep-dive session 2. +0.008, -0.108, -1.788 on the same rows.
I have a placebo, I know its standard deviation, and I can say how many of them the real slope is from zero. Deep-dive session 2 and the bench. 200 within-product shuffles, sd 0.159, the real slope more than 11 away.
I have cut the rows at least three ways and I quote the range those cuts produce, not the number at one setting. The bench. Weighted -2.216, freight -1.881, single-unit months dropped -1.480: between -1.5 and -2.2.
Every revenue projection on the page carries the word "modelled" and the assumption it rests on. Deep-dive session 3. Constant elasticity; a 10 percent rise has never been run on these products.
I can say the scope in one clause: for which products, over which months, on which marketplace. The bench, Part 2. "For the 203 products that repriced, over 21 months, on one marketplace."
I can name the anti-lever when somebody proposes it, and I have the two projected revenue figures side by side. The bench. "Use the overall number": +8.9 percent against -7.2 percent on the same rows.
I have written the sentence with its range, its mechanism and its caveat, and I have said what the next step is and that it is a test, not an extrapolation. The bench, Part 2. Link the experimentation course, do not extrapolate the slope.
Homework

After the bench

30 min
  1. Find the setting that moves the slope most and the one that moves it least. Write both down with their numbers. That pair is the first thing to say when somebody asks how sure you are.
  2. Pick a category and read its slope and standard error. Say whether it is distinguishable from the all-products number, in one sentence, with the standard error as the reason.
  3. Run the anti-lever at -10 percent instead of +10 and read both revenue lines. Write what a price cut is projected to do under each slope, and which one a first analysis would have reported.
  4. Write the sentence for your own catalogue, in the shape from Part 2, with blanks where you do not yet have the number. The blanks are the work plan.
Check yourself

Three questions

Clear all three and deep-dive session 10 gets stamped in your passport.

1 · A colleague proposes pricing on "the overall number, because it uses all the data". What does the bench say?

The anti-lever gets the sign of the decision wrong. Per-product slopes are not the alternative either: a third of them are noise. The panel with product effects is the comparison a pricing decision is about.

2 · What does the break button demonstrate, and what does it not?

A detector that cannot be made to fail has not been tested; the shuffle is the failing state. Removing chance is not the same as removing the seller's choice, which only a price test removes.

3 · Which is the sentence that defends the number?

A range, a mechanism, a caveat, a scope, and the next step. Option A removes the shape; option C has a range with no mechanism, and "0 to -2" includes the anti-lever's number as if it were a candidate.

Sources covered

What this session teaches, and where it came from

✓Every slope, standard error, row count and placebo figure on the benchComputed in the browser from the 2,216-row panel derived from the Brazilian E-Commerce Public Dataset by Olist (Kaggle, CC BY-NC-SA 4.0); the same code agrees with the offline build to three decimals on every deterministic rung, and the placebo to within 0.02
✓The two revenue projectionsModelled under a constant-elasticity assumption and labelled so on the bench
✓Naive standard errors, and the placebo as the honest measure of distance from noiseStandard practice; the page says the errors are computed as if rows were independent
◐Causal identification and the price testNamed, not taught; the causal inference and experimentation courses own them

Every fact on this page, and its verification tier, is recorded in the course's source map.

Deep-dive session 10 cheat sheet · pin this

What was compared with what?The one question. Across products +0.008, pooled -0.108, within product -1.788, with months -1.854.
The range is the four filtersWeighted -2.2, freight -1.9, no single-unit months -1.5. Quote -1.5 to -2.2, about -1.8.
The placebo is the proofShuffle within product: mean -0.002, sd 0.159. The real slope is more than 11 sd away.
The anti-lever"Use the overall number": +8.9 percent projected revenue against -7.2 on the same rows.
Computed vs modelledSlopes are computed. Revenue lines are arithmetic under constant elasticity. Say which.
The sentenceRange, mechanism, caveat, scope, next step. Nobody can take a decimal off it.