learn-economics-with-phoebe / Deep-dive session 5 of 10
Learn Economics with Phoebe · Deep-dive session 5 of 10

Network effects and data moats

Four sessions have priced one product at a time. Tonight is about the claims that get made when a business says its value comes from size: more members, more sellers, more rows. Three of those claims are worth taking apart with arithmetic rather than opinion. The first is that a marketplace runs on a network effect, which is true, and the session says which kind. The second is Metcalfe's law, that value grows with the square of the members, which has a published rebuttal that says n log n, and the two disagree by a factor of seven at one tenfold of members. The third is the data moat, "we have more data so our model is better so we win", which is true on the steep part of a learning curve and quietly stops being true on the flat part. The build-along fits that curve in numpy on constructed data, reads where it flattens, and measures how far behind a rival with a tenth of the rows actually is at two places on it.

🟡 Core Analysts, product and strategy teams Python, numpy 45 min
0-3 · Welcome 3-18 · Two sides, two laws, one curve 18-40 · Build-along: fit the curve, find the flat 40-45 · Q&A
Part 0

Where we start

A network effect is a fact about the demand curve of the previous sessions: the curve for a marketplace shifts right when the marketplace gets bigger, because each member values it more. That is all it is, and it is enough to build a business on. What is not a fact is the exponent. Every "network effects" slide carries an unstated law about how much a doubling is worth, and every "data moat" slide carries an unstated learning curve about how much a doubling of rows is worth. Tonight makes both laws explicit, does the arithmetic where it is arithmetic, and fits the curve where it is a curve. Foundations session 6 met the same ideas without code; the deep-dive does the numbers.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = run the code yourself

★ What you walk out with tonight. The difference between a direct and an indirect network effect and which one a marketplace runs on, the arithmetic of n squared against n log n at n and at 10n done by hand, a learning curve fitted in six lines of numpy with the point where the next doubling buys under one point of accuracy read off it, and one sentence about a data advantage that says how big it is and where on the curve it was measured.

Part 1 · covers direct and indirect effects, the two-sided marketplace

Two kinds of network, and a marketplace with two sides

5 min live

A network effect is direct when a member gains from more members of the same kind, and indirect when a member gains from more members of the other kind. A marketplace is almost entirely the second: a buyer gains from more sellers and a seller gains from more buyers, while a seller gains nothing from another seller except competition. The marketplace dataset behind every session carries 3,095 sellers and 96,096 customer ids, counted in the RFM course on the same export, and the value in it crosses the platform from one side to the other. That is what two-sided means, and it is why the platform's first problem is which side to fill first.

TWO SIDES, ONE PLATFORM 3,095 sellers the RFM course's count 96,096 customer ids same export, same count THE MARKETPLACE 32,216 products with at least one sale listings choice orders, reviews orders, rank more sellers: more choice for every buyer more buyers: more orders for every seller The two long arcs are the indirect effect: each side gains when the other side grows, not when its own does.
🔍 Click to zoom - the short arrows are the trade; the two long amber arcs are the network effect, and neither of them starts and ends on the same side
LiveDirect, indirect, and which one a marketplace runs on3 min▶

Direct. A messaging app is worth more to you when the people you message are on it. Every new member of your own kind makes the thing better for you. The demand curve for the app shifts right with each member, and the members are the same kind of person.

Indirect. A marketplace is worth more to a buyer when there are more sellers, and more to a seller when there are more buyers. Every new member of the other kind makes it better for you; a new member of your own kind is a rival for the same buyers or the same sellers. The two demand curves, one per side, shift right when the other side grows, and that is the whole mechanism of a platform.

The distinction matters for every number that follows. If you count pairs among all members, as Metcalfe's law does, you are counting seller-seller pairs, which carry no value, and buyer-buyer pairs, which carry none either. On a two-sided platform the connections that count cross the middle. With 3,095 sellers and 96,096 customer ids the lopsidedness is the point: a seller's network is about 31 buyers on average, and a buyer's network is the sellers. The average is pure arithmetic on the two counts; what any one of those connections is worth is the claim Part 2 takes apart.

Self-studyThe chicken-and-egg problem in one page3 min▶

An indirect network effect has a cruel start. Nobody sells where nobody buys, and nobody buys where nobody sells, so a two-sided platform at birth has two empty sides and no reason for either to fill. Every platform that exists solved this somehow, and the solutions all amount to subsidising one side until the other side has a reason to arrive: free listings for sellers, a discount for the first buyers, a supply the platform provides itself, or a single category served so well that both sides show up for it before the platform is general.

The data habit: when a platform's early numbers look strange, one side growing while the other is flat, or a category dominating the whole, the strangeness is usually the subsidy, not a finding. The Olist panel's 21 months run from October 2016 with gaps, and the early months are thin on both sides; that is what the start of a two-sided platform looks like in rows, and it is one reason the elasticity sessions asked for six months with a sale before reading any product.

Real world

Two sides in one export. The Olist dataset carries 3,095 sellers, 96,096 customer ids and 32,216 products with at least one sale; the RFM course counted the first two on the same export and the elasticity sessions counted the third. Only 203 of those products can be read for a price response, which says something about the seller side that the count alone does not: most sellers listed, sold a little, and never repriced. A platform's "network" is often mostly quiet members, and the arcs in the figure are drawn by the ones who trade.

Part 2 · covers Metcalfe as a claim, the n log n rebuttal

n squared, n log n, and the arithmetic at 10n

5 min live

Metcalfe's law says the value of a network grows with the square of its members, because that is how the number of possible pairs grows. Briscoe, Odlyzko and Tilly published a rebuttal in IEEE Spectrum in 2006, titled "Metcalfe's Law is Wrong", arguing that value grows in proportion to n log n, because a member's connections are not all worth the same. Both are claims. The useful thing to do with two claims about the same quantity is to compute both at two sizes and see how far apart they are. At n and at 10n they are apart by a factor of seven, and that factor is a valuation argument in one number.

TWO LAWS, ONE TENFOLD OF MEMBERS both axes are log scale, so each claim is a straight line VALUE, log scale MEMBERS n = 1,000 10n = 10,000 n squared n log n the square: every pair counts n log n: the k-th link is worth 1/k THE ARITHMETIC at n = 1,000 n squared1,000,000 n ln n6,908 at 10n = 10,000 n squared100,000,000 n ln n92,103 n squared grew 100x. n log n grew 13.3x. Log base does not change the ratio. Two claims about one tenfold of members. Briscoe, Odlyzko and Tilly 2006: value grows as n log n, because a member's links are not all worth the same.
🔍 Click to zoom - the two amber dots are the same ten thousand members under two laws, a hundred times and 13.3 times, and the gap between them is a valuation
LiveThe arithmetic, done once by hand so nobody has to trust a slide3 min▶

Take n = 1,000. n squared is 1,000,000. n ln n is 1,000 times 6.908, which is 6,908. Now take 10n = 10,000. n squared is 100,000,000, a hundred times more. n ln n is 10,000 times 9.210, which is 92,103, and 92,103 over 6,908 is 13.3. So a tenfold in members is a hundredfold in value under one law and a 13-fold under the other, and the two disagree by a factor of 7.5 on the same tenfold. The base of the logarithm does not matter: base 2 or base 10 scales both n log n figures by the same constant, and the ratio 13.3 survives.

Why 13.3 and not 10: the log term itself grows, from 6.908 to 9.210, so n log n grows a little faster than n. That is the whole of the rebuttal's shape. Value grows faster than the member count, so a bigger network really is worth more per member, but not much faster, and nothing like the square.

What neither number is: a measurement. Nobody has measured the exponent on the marketplace whose rows run through these sessions, and the map of every number on these pages says so. When a deck says "network effects" and draws a curve, the honest question is which of the two lines in the figure it drew, and what it would take to tell them apart. A doubling gives 4x under the square and 2.2x under n log n; a tenfold gives 100x against 13.3x. Ask which one the forecast assumed.

Self-studyWhy n log n: the argument about the k-th most valuable link2 min▶

Metcalfe's square rests on one assumption: every possible connection is worth the same. The rebuttal replaces it with a ranking. Each member has a most valuable connection, a second most valuable, and so on, and Briscoe, Odlyzko and Tilly's argument is that the k-th one is worth about 1/k of the first, the way the k-th most common word or the k-th biggest city is about 1/k of the first. Summing 1 + 1/2 + 1/3 + ... + 1/n gives about ln n. So each member gets about log n of value, and n members get about n log n.

The marginal reading is the useful one. Under the square, the thousandth member adds about as much as the first, because they add a thousand new pairs. Under n log n, the thousandth member's own connections are worth log n, and they add a little to everybody else's log n, which is much less than a thousand equal pairs. That is why the two laws diverge so fast, and why the argument between them is really an argument about whether a new member's links are worth the same as an old member's.

Real world

A constructed deck, with the two lines drawn on it. Imagine a platform pitch that shows members going from 1,000 to 10,000 in three years and says value will follow "by Metcalfe's law". Draw the n log n line on the same slide. The square promises a hundredfold; the rebuttal promises about thirteen. Everything in the deck that depends on the hundredfold, the valuation, the hiring plan, the price of the next round, is standing on the difference between those two lines, and neither has been measured for this platform. The deck is invented. The gap between the lines is arithmetic.

Part 3 · covers data network effects, learning curves, switching costs

When more data compounds, and when it plateaus

5 min live

The data moat is a loop: more customers give more data, more data gives a better model, a better model wins more customers. The loop is real at the start. It stops being a moat for a mechanical reason: each doubling of data buys less accuracy than the doubling before it, because the early rows teach the common cases and the later rows are mostly common cases again. That is a learning curve that flattens, and once an incumbent and a rival are both on the flat part, a rival with a tenth of the rows is a few points behind rather than a generation behind. The build-along constructs a series with that shape, fits it, and reads where the flat begins.

THE CURVE THAT FLATTENS constructed series: error falls as a power of the rows, plus noise ACCURACY, percent 1k 4k 16k 64k 256k 512k ROWS, each step doubles +7.7 +5.9 +4.5 +3.4 +2.6 +1.9 +1.5 +1.1 +0.8 READ THE STEPS each doubling buys less 7.7, 5.9, 4.5, 3.4, 2.6, 1.9, 1.5, 1.1, then 0.8 points the flat starts where a doubling buys under a point: here, at 256,000 rows every number is constructed the shape is the mechanism What remains to learn after many rows is the rare cases; the next batch of rows is mostly the common cases again.
🔍 Click to zoom - the hatched band is the flat: from there a doubling of rows buys under one point, and a rival with a tenth of your data is standing on the same shelf
LiveThe learning curve flattens, and the rival's gap shrinks with it3 min▶

Draw accuracy against rows, with each step on the horizontal axis a doubling. The first doublings buy a lot: the model is learning the common cases, and every batch of rows is full of them. Later doublings buy less, because the common cases are learned and what is left is rare, and a batch of new rows contains the rare cases at the same low rate as before. On the constructed curve in the figure the nine doublings buy 7.7, 5.9, 4.5, 3.4, 2.6, 1.9, 1.5, 1.1 and 0.8 points; the exact numbers are invented, the shape is what every learning curve shares. The published literature on this shape reports a range of exponents that differ by task, and none of them is quoted here because none has been checked at source for these pages.

Now put a rival on the same curve with a tenth of your rows. On the steep part the gap is large: on the constructed curve, 1,000 rows against 10,000 is about 19 points apart. On the flat part it is small: 25,600 rows against 256,000 is about 5 points apart. Same tenfold advantage in data, a quarter of the advantage in accuracy, and the gap keeps shrinking as both of you reach the flat. That is what "we have ten times the data" is worth, and it depends entirely on where on the curve it is measured.

The same flattening lives inside the elasticity sessions in another costume. One product's 13 rows gave a slope of -10.17 that was noise; the panel of 2,216 rows gave -1.788 with a naive standard error of 0.149. More rows turned noise into a range, which is the steep part. More rows after that narrow the standard error at a square-root pace, which is the flat part: the answer stops changing, it just gets a tighter interval, and a rival with a smaller panel gets nearly the same answer.

Self-studyWhy "we have the data" is often not a moat: switching costs and multi-homing3 min▶

A data advantage is a moat under two conditions, and the learning curve is why. Either the rival cannot get to the steep part at all, because the data is genuinely unobtainable: it is produced by a process only you run, or it is tied to customers who would have to leave you to produce it for somebody else. Or the curve for your task has not flattened at the sizes anyone has, so every doubling still pays. The first condition is a switching cost wearing a data costume: the data is a moat because the customer is walled in, not because of the rows. The second is rare and temporary, since the curve flattens eventually for every task anyone has measured.

Where neither condition holds, the moat claim fails on arithmetic. A rival with public data, or with a tenth of yours bought or collected, sits a few points behind on the flat, and a few points is what a better model architecture, a cleaner label set or a lower price can close. And where customers multi-home, using your product and the rival's at once, they produce data for both, and the rival's curve climbs on your customers. Foundations session 6 drew the wall and the door; the data version is that the rows are the bricks, and the door is that the customer can produce the same rows for somebody else.

The honest sentence for a leader has three parts: how big the advantage is in points, where on the curve it was measured, and what would let a rival reach the same shelf. A sentence with only the first part is a slogan.

Real world

A constructed model team, and the number they did not ask for. Imagine a team with 256,000 labelled rows and a rival with 25,600. The team's slide says "ten times the data". On the constructed curve the accuracy gap at that point is about 5 points, and the next doubling of their own rows would buy under one. Their advantage is real, measured, and standing on the flat part of the curve, where the rival's next doubling buys more than theirs does. The team is invented, the curve is constructed, and the sentence they should have written is the one the build-along prints: how big, where on the curve, and what closes it.

Build-along

Fit a learning curve to constructed data, and read where it flattens

★ 22 min · everyone builds

Six steps in numpy. The data is constructed: a series of accuracy readings at doubling row counts, with the error falling as a power of the rows plus a little noise, which is the shape learning curves share. Nothing here is measured on any real model, so every printed output on this page says "your numbers will differ", and the constants at the top are yours to change. The point is the reading, not the numbers: a fit in logs, the points each doubling buys, the first doubling that buys under one, and a rival's gap measured at two places on the curve.

SIX STATIONS every number produced along the track is constructed 1 · construct rows x2 nine times error falls, noisy 2 · look print the table numbers differ 3 · fit in logs log err vs log n slope = exponent 4 · doublings points a doubling buys; first under 1 5 · rival's gap tenth of the rows early vs late 6 · the sentence flat starts at n gap shrinks Station 1 is the only one that knows the true curve. Stations 3 to 6 see only the noisy readings, which is what a real project sees. Change the two constants at the top and run it again. The flat moves; the reading does not.
🔍 Click to zoom - the hatched station is the only one that knows the truth; the amber station is the one that makes the moat claim measurable

Construct the series. Rows double nine times from 1,000 to 256,000. The true error falls as a power of the rows, which a real project never sees; what it sees is the noisy accuracy readings. Every constant is labelled and yours to change.

★ Step 1 · construct, and say soimport numpy as np # CONSTRUCTED: an accuracy-vs-rows series with diminishing returns. # Nothing below is measured on any real model. Change the constants freely. SEED = 20260925 START = 0.32 # error at 1,000 rows DECAY = 0.40 # how fast error falls: the exponent on the rows NOISE = 0.006 # measurement noise on each accuracy reading rng = np.random.default_rng(SEED) rows = 1000 * 2 ** np.arange(9) # 1,000 to 256,000, one doubling per step err = START * (rows / 1000) ** (-DECAY) # the true curve; a real project never sees it acc = 1 - err + rng.normal(0, NOISE, len(rows)) # what a real project sees: noisy readings

Look before fitting. Print the table and read the steps by eye first. The gaps between consecutive readings should shrink; if they do not, the noise is too large for the series and the fit will say so later.

★ Step 2 · lookfor n, a in zip(rows, acc): print(f"{n:>8,} rows accuracy {100 * a:5.1f}") # constructed; your numbers will differ

Fit a line in logs. A straight line through log error against log rows is a power law in levels, which is a saturating curve in accuracy. The slope is the exponent, the thing the moat claim quietly depends on. It is the same trick as the elasticity fit: a slope in logs is a percentage answer.

★ Step 3 · fit in logs# log(error) = c + b * log(rows): a straight line in logs, a power law in levels b, c = np.polyfit(np.log(rows), np.log(1 - acc), 1) print(round(b, 3)) # the fitted exponent; near -DECAY here, your numbers will differ def fitted_err(n): return np.exp(c) * n ** b

Read what each doubling buys. Extend the fitted curve past the data and compute the points of accuracy each doubling adds. The first doubling that buys under one point is where the flat begins, and it is the number to write on the slide instead of "we have the data".

★ Step 4 · the doublingsgrid = 1000 * 2 ** np.arange(12) # 1,000 to 2,048,000 rows gain = 100 * (fitted_err(grid[:-1]) - fitted_err(grid[1:])) # points bought by each doubling for n, g in zip(grid[:-1], gain): print(f"{n:>10,} to {2 * n:>10,} +{g:4.2f} points") # your numbers will differ flat = grid[:-1][gain < 1][0] print("first doubling that buys under one point starts at", f"{flat:,}", "rows")

Measure the rival's gap at two places. A rival with a tenth of the rows is a fixed ratio behind in data and a shrinking distance behind in accuracy. Compute the distance early on the curve and at the flat. On the noiseless constructed curve those are about 19 points at 10,000 rows and about 5 at 256,000; your fitted values will sit near them.

★ Step 5 · the rival's gapdef gap_points(n_big): # how far behind a rival with a tenth of the rows sits, in points, at this place on the curve return 100 * (fitted_err(n_big / 10) - fitted_err(n_big)) print(round(gap_points(10_000), 1), round(gap_points(256_000), 1)) # your numbers will differ # same tenfold in data, a fraction of the gap in accuracy: the moat is where on the curve you stand

Write the sentence. Three parts: how big, where on the curve, and what it means for a rival. Print it from the fitted numbers so that changing a constant changes the sentence and not just a chart.

★ Step 6 · the sentenceprint(f"On this constructed series, accuracy stops paying for rows at about {flat:,}: " f"the next doubling buys under one point. A rival with a tenth of our rows is " f"{gap_points(flat):.0f} points behind there, against {gap_points(10_000):.0f} points " f"behind at 10,000 rows. The advantage shrinks as both of us reach the flat.") # your numbers will differ; the shape of the sentence should not

Keep the two constants at the top and the sentence at the bottom. When somebody says "but our task is different", change the decay and run it again. A steeper decay moves the flat earlier and shrinks the rival's gap faster; a shallower one moves it later. Either way the sentence has the same three parts, and a slide that cannot fill all three is a slogan.

Homework

Before deep-dive session 6

30 min
  1. Run the six steps and then change the decay to 0.25 and to 0.60. Write down where the flat starts and the rival's gap at the flat for each. Three rows, and you have the whole range a moat claim can honestly occupy on this shape.
  2. Do the n and 10n arithmetic for a platform you know, with n set to its real member count if you have it. Write the n squared and n log n values at n and at 10n and the two ratios. Then write which one the platform's own forecast assumed.
  3. Find a real learning curve in a paper or a model card, if you can, and read whether the last doubling shown bought more or less than one point. Write one sentence on where on the curve the headline number was measured.
  4. Bring the sentence. Session 6 is auctions, and the first thing an ad auction prices is a slot on a platform whose value somebody once quoted as a square.
Check yourself

Three questions

Clear all three and deep-dive session 5 gets stamped in your passport.

1 · A marketplace with 3,095 sellers and 96,096 customer ids says it has "strong network effects". Which kind, and what follows for counting them?

A marketplace is two-sided. Seller-seller pairs are rivalry, not value, and buyer-buyer pairs are nothing. The average of about 31 buyers per seller is arithmetic on the two counts; what one of those connections is worth is a claim, and the chicken-and-egg problem at birth is why the early rows of any platform look strange.

2 · A platform goes from 1,000 to 10,000 members. What do the two laws say about its value, and what is the honest reading?

The arithmetic is the whole answer, and it is the same in any log base. The rebuttal, Briscoe, Odlyzko and Tilly 2006, does not say value falls; it says value grows faster than members but nothing like the square, because a member's k-th link is worth about 1/k of the first.

3 · A team has ten times the training rows of its rival and says that is its moat. What does the learning curve say?

On the constructed curve the gap is about 19 points at 10,000 rows and about 5 at 256,000, a quarter of the advantage for the same tenfold. A switching cost or unobtainable data can make a data advantage durable, but that is a wall, not the rows, and the door question from Foundations session 6 applies.

Sources covered

What this session teaches, and where it came from

✓Direct and indirect network effects, the two-sided platform, switching costs, multi-homingStandard economics, no single source
✓3,095 sellers and 96,096 customer ids; 32,216 products with a sale; 203 eligible; 2,216 panel rows over 21 months; rung 3 slope -1.788, naive se 0.149; the headline product's -10.17Counted from the Brazilian E-Commerce Public Dataset by Olist (Kaggle, CC BY-NC-SA 4.0); the seller and customer counts are the RFM course's on the same export, the rest computed for these pages
✓Metcalfe's law as a claim; the n log n rebuttal; the arithmetic at n = 1,000 and 10n = 10,000 (100x against 13.3x), and at one doubling (4x against 2.2x)Briscoe, Odlyzko and Tilly 2006, IEEE Spectrum, "Metcalfe's Law is Wrong", read at source; the figures are arithmetic on the two claims, not a measurement of any platform
✓The learning curve that flattens; the rival's gap of about 19 points at 10,000 rows and about 5 at 256,000; the flat at 256,000 rowsConstructed series (error 0.32 at 1,000 rows, exponent 0.40, noise 0.006); every number is invented and the page says so; no published exponent is quoted
✓The constructed deck and the constructed model teamInvented; the page says so
◐The chicken-and-egg problem, subsidising one sideSelf-study, one page; no case figures claimed
◐Auctions, Vickrey, GSPNamed as deep-dive session 6; not taught here

Every fact on this page, and its verification tier, is recorded in the course's source map.

Deep-dive session 5 cheat sheet · pin this

Direct or indirectGain from your own kind, or from the other side. A marketplace is indirect: pairs on one side are rivals, not value.
The arithmetic at 10nn squared: 1,000,000 to 100,000,000, 100x. n ln n: 6,908 to 92,103, 13.3x. Log base does not change the ratio.
Both laws are claimsThe mechanism is real, the exponent is not measured. Ask which line the forecast drew.
Fit in logspolyfit(log rows, log error, 1). The slope is the exponent; the curve is a power law, a saturating curve in accuracy.
The flat is where a doubling buys under one pointRead it off the fitted curve. On the constructed series, 256,000 rows.
A data advantage has three partsHow big in points, where on the curve, what lets the rival reach the same shelf. Constructed: 19 points early, 5 on the flat.