learn-data-modeling-with-phoebe / Builder session 3 of 10
Learn Data Modeling with Phoebe · Builder track · Session 3 of 10

ER, cardinality and keys

Session b1 gave you eight entities. Session b2 gave you a reason to keep every fact in one place. This session turns that list into a logical model: read the cardinality off the verb, resolve the two many-to-many relationships with associative tables, and then make the hardest small decision in modeling - which key. Bazaar splits it three ways on purpose, and by the end you will be able to defend each split in one sentence.

🟡 Builder track · medium Analysts · analytics engineers · DE · DS Runs in your browser · real SQLite 45 min
0-3 · Welcome 3-15 · Cardinality, read off the verb 15-40 · Build-along: resolve it, then key it 40-45 · Q&A
Part 0

From a list of things to a model

A conceptual model is a list of nouns and a handful of lines. A logical model is that same picture with three extra facts written on it: how many of each side participate in the relationship, whether participation is optional or mandatory, and what identifies a row. Those three additions are all this session is. They are also where most schemas quietly go wrong, because each one looks like a formality and each one is actually a business rule.

Everything here is still engine-independent. No types, no indexes, no DDL - those are session b4, which takes the model you finish today and ships it as sql/01_oltp_ddl.sql. What you decide now is what that file will be forced to say.

Live - presented in session Self-study - read after class ▶ Live SQL - editable & runnable Sources covered
★ What you walk out with today The habit of reading cardinality straight out of the sentence the business already says, two resolved many-to-many relationships with their own attributes, and a defensible answer to "why is this key an integer and that one a string" for every table in Bazaar.
Part 1 · covers Codd / relational fundamentals, Fowler and Evans on naming

Cardinality is already in the verb 6 min live

You do not need a notation to find cardinality. You need the sentence the business says out loud, and then two questions about it: how many? and must there be one? "A shopper places orders" is one-to-many. "An order must have a shopper" is mandatory. "A cart may become an order" is optional, and that single word may is why the cart column on an order is allowed to be empty while the user column is not.

Then there is the third shape, the one that cannot be stored as written: many-to-many. "A cart holds many products, and a product sits in many carts" is true, useful, and impossible to put in two tables. It needs a third one - and that third table turns out to be the most interesting table in the schema.

Before - the sentence the business says carts 2,229 rows products 60 rows many carts hold many products and back again, both directions A many-to-many cannot be stored as drawn. No column on carts and no column on products can hold "two of this product, in that cart, added at 15:51" - the fact belongs to the pair, not to either side. After - resolved by an associative table carts one per cart cart_items 3,246 rows products one per product one cart holds many cart items each cart item names exactly one product The associative table earns its keep by carrying facts of its own: quantity, added_ts, and on an order line also line_no, unit_price and discount_amt. If it has nothing of its own to store, it is only plumbing. Its primary key states the grain out loud: (cart_id, product_id) is one row per product per cart.
🔍 Click to zoom - the move you will make twice today: many-to-many becomes two one-to-many relationships plus a table with its own attributes
LiveProve the many-to-many with two queries4 min

A many-to-many is not a design opinion. It is a fact you can measure, and it is only real if it holds in both directions. One product in many carts is half the claim; one cart holding many products is the other half. Run both.

SELECT p.product_id, p.product_name, m.merchant_id,
       COUNT(DISTINCT ci.cart_id) AS carts_holding_this_product
FROM cart_items ci
JOIN products  p ON p.product_id  = ci.product_id
JOIN merchants m ON m.merchant_id = p.merchant_id
GROUP BY p.product_id, p.product_name, m.merchant_id
ORDER BY carts_holding_this_product DESC
LIMIT 5;

SELECT ci.cart_id, COUNT(*) AS products_in_this_cart,
       GROUP_CONCAT(ci.product_id) AS product_ids
FROM cart_items ci
GROUP BY ci.cart_id
HAVING COUNT(*) = 3
ORDER BY ci.cart_id
LIMIT 5;

The Nimbus Audio products sit at the top of the first result because M07 is Bazaar's biggest seller for most of the quarter - a fact sessions b8 and b9 spend their time on. For now the only thing that matters is the shape: a product appears in many carts, a cart holds many products, so neither table can own the relationship.

The one-direction test If the relationship is only "many" on one side, you do not need a third table. A merchant sells many products, but a product belongs to exactly one merchant - so products.merchant_id is enough, and inventing a merchant_products table would add a join for nothing.
LiveOptional versus mandatory, and the column that proves it3 min

Participation is the half of cardinality people skip, and it is the half that decides whether a column may be empty. Two sentences from Bazaar, one word apart:

  • "An order must have a shopper." Mandatory. orders.user_id is required, and no order can exist without one. Making that column optional would let an order arrive with nobody to ship to, invoice, or apologise to.
  • "A cart may become an order." Optional. Most carts never do - 985 orders against 2,229 carts. So orders.cart_id is allowed to be empty, because an order can also be placed by a route that never created a cart row, and a cart with no order is the normal case rather than an error.

Write both sentences down before you write the column. "Must" becomes a required column; "may" becomes an optional one. When somebody later asks why one is nullable and the other is not, the answer is a business rule, not a preference - and that is exactly the difference between a model and a schema that grew.

Self-studyReading a relationship you did not design3 min read

When you inherit a schema, the cardinality is already decided and usually undocumented. You can recover it in three queries per relationship, and it is worth doing before you trust any join:

  • Is the child side really "many"? Group the child table by the foreign key and look at the maximum count. If it is always 1, somebody modelled a one-to-one as a one-to-many, and the join is safe but the design is misleading.
  • Is participation mandatory? Count rows where the foreign key is empty. If the count is zero but the column is nullable, you have a rule the application enforces and the database does not - which lasts until the first backfill script.
  • Is the parent side really "one"? This is the one people miss. If two parent rows can match one child through the join column, your "one-to-many" is a many-to-many in disguise and every aggregate over that join is double-counting.

That third check is the source of an enormous share of "the numbers do not match" incidents. A join that fans out is not a broken query; it is an unmodelled many-to-many being discovered at report time.

RelationshipRead it as a sentenceCardinalityHow the model resolves it
merchants - productsa merchant sells many products; a product belongs to exactly one merchantone-to-many, mandatory both sidesproducts.merchant_id, required
users - cartsa shopper creates many carts; every cart belongs to one shopperone-to-many, mandatorycarts.user_id, required
users - ordersan order must have a shopperone-to-many, mandatoryorders.user_id, required
carts - ordersa cart may become an orderzero-or-one to one, optionalorders.cart_id, allowed to be empty
carts - productsa cart holds many products; a product sits in many cartsmany-to-manycart_items, key (cart_id, product_id)
orders - productsan order sells many products; a product is sold on many ordersmany-to-manyorder_items, key (order_id, line_no)
orders - order_itemsan order has many lines; every line belongs to one orderone-to-many, mandatoryorder_items.order_id, part of the key
orders - transactionsan order receives zero, one or several payment attemptsone-to-many, optional on the attempt sidetransactions.order_id, required
Part 2 · covers Codd on keys and functional dependency, Kimball on grain

Natural, surrogate, composite 6 min live

A key answers one question: what makes this row this row and not another one. Bazaar answers it three different ways, and the split is not inconsistency - each table is telling you who owns its identity. merchants.merchant_id is a natural key because the business genuinely owns it: 'M07' is meaningful, stable, and printed on invoices. users.user_id and orders.order_id are surrogate integers, because email addresses change and nothing outside the system owns an order number. cart_items and order_items use composite keys, because the thing being identified is a pair.

Natural key the business owns it merchants.merchant_id 'M07' is meaningful, stable and printed on invoices. Support reads it aloud on the phone. Risk: if the business ever renumbers, you renumber too Surrogate key the system owns it users.user_id, orders.order_id Email changes, and nothing outside Bazaar owns an order number. So the id is an integer with no meaning. Risk: meaningless to humans, so print something else too Composite key the pair is the row cart_items(cart_id, product_id) order_items(order_id, line_no) The key is the grain sentence, written where the database can enforce it. Risk: every child table has to carry both halves forward The question is never "natural or surrogate" in the abstract. It is: does anyone outside this database already own this identifier, and will they keep it stable? Merchant codes, country codes and currency codes qualify. Email addresses, phone numbers and people's names never do.
🔍 Click to zoom - three key strategies, one question: who owns the identifier
LiveWatch the surrogate key do its job4 min

The argument for surrogate keys is usually made in the abstract. Here it is as a query: change a shopper's email address, which is the natural candidate key everyone reaches for first, and count what is still attached afterwards.

SELECT m.merchant_id, m.merchant_name, p.sku, p.product_name
FROM products p
JOIN merchants m ON m.merchant_id = p.merchant_id
WHERE m.merchant_id = 'M07'
ORDER BY p.product_id;

-- the shopper changes their email address, which happens constantly
UPDATE users SET email = 'newaddress001@example.com' WHERE user_id = 1;

SELECT u.user_id, u.email,
       (SELECT COUNT(*) FROM orders o WHERE o.user_id = u.user_id) AS orders_still_attached,
       (SELECT COUNT(*) FROM carts  c WHERE c.user_id = u.user_id) AS carts_still_attached
FROM users u
WHERE u.user_id = 1;

Nothing broke, and nothing had to be updated anywhere else. Now imagine email had been the primary key: that one address change becomes a cascading update across carts, orders and every downstream table that ever copied it, and any row a cascade misses becomes an orphan. Email is still UNIQUE in Bazaar, because two accounts must not share one - a uniqueness rule and a primary key are different jobs, and confusing them is what produces primary keys made of people's contact details.

The first result set shows the other side of the coin. M07 travels inside every SKU (M07-1031), and it appears on invoices and in support conversations. That is a natural key genuinely doing work, so Bazaar keeps it as the primary key of merchants rather than hiding it behind an integer nobody can read.

LiveA composite key is a statement about grain3 min

Session b1 asked you to write the grain of every table as a sentence. A composite primary key is that sentence, in a form the database enforces:

  • cart_items (cart_id, product_id) says "one row per product per cart". Adding the same product to the same cart twice is not two rows - it is one row with a bigger quantity. The key makes that a rule instead of a convention.
  • order_items (order_id, line_no) says "one row per line on an order". Note what it does not say: the same product may legitimately appear on two different lines of one order, at two different prices, because that is how a promotion or a partial substitution looks. Keying on (order_id, product_id) would forbid a real business event.

Those two keys are different on purpose, and the difference is a business decision rather than a stylistic one. Cart lines merge; order lines do not, because an order line is a historical record of what was charged and merging two of them would rewrite history.

The grain test for any key Read the primary key out loud as "one row per ...". If the sentence is true and complete, the key is right. If you need the word "and" in an unexpected place, or you cannot finish the sentence at all, the key is not describing the grain you actually have.
Self-studyIdentifiers at scale: identity columns and UUIDv73 min read

Bazaar writes INTEGER PRIMARY KEY because SQLite is what runs in your browser tab. Two honest notes for when you ship something real:

  • On Postgres, use GENERATED ALWAYS AS IDENTITY. It is the ANSI SQL way to say "the database owns this number", and unlike the older SERIAL pattern it refuses inserts that try to supply the value by hand. That refusal is the point: a surrogate key nobody outside the database may set is a surrogate key that cannot drift.
  • With more than one writer, use UUIDv7. A single auto-increment counter needs a single source of truth, which is exactly what you do not have across shards, regions or offline clients. Random UUIDv4 solves collisions but destroys locality: inserts scatter across the index and reads that want "the newest orders" have to look everywhere. UUIDv7 puts a timestamp in the high bits, so identifiers stay roughly sortable by creation time and both problems go away at once.

What does not change with scale is the reasoning. The key still has to be stable, meaningless to the business unless the business genuinely owns it, and never something a human types twice. Bigger systems change the mechanism, not the question.

Demo 1 of 2

Build the associative table yourself ★ 12 min · everyone builds

Two steps. First measure what the composite key is claiming about Bazaar's real data, then build the same table from scratch in an empty database and watch the key refuse the row it is designed to refuse. The second editor starts with nothing in it, so you write the DDL as well as the queries.

Count three ways. Rows, distinct cart ids, distinct pairs. If distinct pairs equals rows, the pair is unique. If distinct cart ids is smaller than rows, cart_id alone is not.

Look for a repeat. Ask the data whether any (cart_id, product_id) pair is stored twice. The answer is zero - that is the key doing its job before you ever declare it.

Then build it. Create the table, insert two products into one cart, and try to add the first product again. Watch which rows survive.

SELECT COUNT(*)                                   AS rows_in_cart_items,
       COUNT(DISTINCT cart_id)                     AS distinct_cart_ids,
       COUNT(DISTINCT product_id)                  AS distinct_product_ids,
       COUNT(DISTINCT cart_id || '-' || product_id) AS distinct_pairs
FROM cart_items;

SELECT COUNT(*) AS pairs_stored_more_than_once
FROM (SELECT cart_id, product_id FROM cart_items
      GROUP BY cart_id, product_id
      HAVING COUNT(*) > 1);
CREATE TABLE cart_items_demo (
  cart_id     INTEGER NOT NULL,
  product_id  INTEGER NOT NULL,
  quantity    INTEGER NOT NULL CHECK (quantity > 0),
  added_ts    TEXT    NOT NULL,
  PRIMARY KEY (cart_id, product_id)
);

-- two different products in the same cart: both allowed, the pair differs
INSERT INTO cart_items_demo VALUES (9001, 1032, 2, '2026-06-01 10:15:00');
INSERT INTO cart_items_demo VALUES (9001, 1033, 1, '2026-06-01 10:16:00');

-- the same product again in the same cart: the pair already exists, so it is refused
INSERT OR IGNORE INTO cart_items_demo VALUES (9001, 1032, 5, '2026-06-01 10:20:00');

SELECT cart_id, product_id, quantity, added_ts
FROM cart_items_demo
ORDER BY product_id;

Two rows, not three. INSERT OR IGNORE is what lets you watch the rejection without the whole script stopping - drop the OR IGNORE and re-run to see the hard error instead, which is how a real application would experience it. Either way the database, not the application, is the thing enforcing "one row per product per cart". Change the third insert to product 1034 and re-run: now three rows land, because the pair is new.

Real world

The associative table that was allowed to be a bag. A retailer's basket table had no primary key at all - just cart_id, product_id and quantity, with the application "making sure" it never inserted a duplicate. A retry storm during a checkout outage inserted duplicates for about forty minutes. Nobody noticed until a merchandising report showed basket sizes had jumped, and by then the duplicates were spread through three downstream tables. The fix was one line of DDL that had been missing since the beginning, plus two weeks of cleanup. A composite primary key on an associative table is not paperwork; it is the only thing standing between a retry and a wrong number.

Demo 2 of 2

Your turn: the header and its lines ★ 10 min · build your own

The header and line pair is the shape you will meet most often in your career: orders and order lines, invoices and invoice lines, shipments and shipment lines, claims and claim lines. Get it right once and you get it right forever, so it is worth proving to yourself rather than accepting.

LiveQ1 · What the associative table is carrying4 min

Take the fullest cart in Bazaar and look at what cart_items stores that neither carts nor products could. Then look at the quantity distribution across all 3,246 cart lines.

SELECT ci.cart_id, ci.product_id, p.product_name, ci.quantity, ci.added_ts
FROM cart_items ci
JOIN products p ON p.product_id = ci.product_id
WHERE ci.cart_id = (SELECT cart_id FROM cart_items
                    GROUP BY cart_id
                    ORDER BY COUNT(*) DESC, cart_id
                    LIMIT 1)
ORDER BY ci.product_id;

SELECT quantity, COUNT(*) AS cart_lines
FROM cart_items
GROUP BY quantity
ORDER BY quantity;

Try the thought experiment the second result makes concrete: to store those quantities without an associative table you would need a column per product on the cart, sixty columns wide, all empty except two or three - and a schema change every time a merchant launches something. That is what "resolve the many-to-many" is buying you.

LiveQ2 · Why quantity cannot live on the order4 min

The header carries what is true of the whole order: who placed it, when, where it ships, in what currency. The lines carry what is true of one product on it. Find orders where the quantities differ line by line and the argument makes itself.

SELECT oi.order_id, oi.line_no, oi.product_id, oi.quantity, oi.unit_price
FROM order_items oi
WHERE oi.order_id IN (SELECT order_id FROM order_items
                      GROUP BY order_id
                      HAVING COUNT(*) > 1 AND COUNT(DISTINCT quantity) > 1
                      ORDER BY order_id
                      LIMIT 3)
ORDER BY oi.order_id, oi.line_no;

SELECT COUNT(DISTINCT order_id) AS orders,
       COUNT(*)                 AS order_lines,
       ROUND(1.0 * COUNT(*) / COUNT(DISTINCT order_id), 2) AS avg_lines_per_order,
       MAX(line_no)             AS most_lines_on_one_order
FROM order_items;

985 orders, 1,434 lines, about 1.5 lines per order. That ratio is small enough to tempt somebody into flattening the two tables together "because most orders only have one line" - and it is exactly the temptation that produces the dashboard from session b1's quiz, where 1,434 lines get reported as 1,434 orders. The header and the line are two grains. Two grains, two tables.

Self-studyQ3 · Draw one relationship from your own schema3 min

No SQL. Pick one relationship you work with daily and write it out in four lines, exactly as Bazaar's table above does it:

  • The sentence the business says, using the business's own words for both nouns.
  • The cardinality: one-to-many, many-to-many, or one-to-one.
  • Whether each side is optional or mandatory, and therefore which column may be empty.
  • What identifies a row on each side, and whether that identifier belongs to the business or to the database.

Then check one thing in the actual data: does the join ever fan out more than you expect? If a relationship you believe is one-to-many turns out to have two matching parents for some child, you have found an unmodelled many-to-many, and every aggregate anyone has ever run across that join has been double-counting. Bring the case to b4, where the constraints that would have prevented it get declared and enforced.

Homework

Try it yourself - this week ◐ 20-30 min total

Source material

Sources covered

Full source map, including what is deliberately out of scope, in materials/official-course-map.md. This page covers:

Codd / relational fundamentals - candidate keys, primary keys, functional dependency, resolving many-to-manyParts 1 and 2 · measured on real data rather than defined
Kimball & Ross - grain as the first design decisionPart 2 · the composite key read out loud as the grain sentence
ANSI SQL plus the PostgreSQL and SQLite documentation - identity columns, uniqueness versus primary keySelf-study · GENERATED ALWAYS AS IDENTITY and UUIDv7; constraint semantics are session b4
Fowler (PoEAA) and Evans (DDD) - ubiquitous language, naming as a modeling decisionPart 1 · the verb in the business's own sentence carries the cardinality
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · A cart holds many products and a product sits in many carts. What does the model need?

A many-to-many cannot be stored in two tables, because the interesting fact belongs to the pair rather than to either side. The associative table is where quantity and the time it was added finally have somewhere to live. Option A also breaks 1NF, which session b2 covered.

2 · Why is merchants.merchant_id a natural key while orders.order_id is a surrogate integer?

The test is ownership and stability: does someone outside this database already own the identifier and keep it stable. Merchant codes qualify, and M07 shows up inside every SKU and on invoices. An order number has no owner outside the system, so the system owns it.

3 · cart_items is keyed on (cart_id, product_id) but order_items is keyed on (order_id, line_no). Why the difference?

Cart lines merge, because adding a product twice means one line with a bigger quantity. Order lines do not, because a line is a historical record of what was charged and a promotion or substitution can produce two lines for the same product. The two keys encode two different business rules.

Builder session 3 cheat sheet · pin this

Cardinality is in the verb"A shopper places orders" is one-to-many. Read the sentence, then ask how many and must there be one.
May versus must"A cart may become an order" makes orders.cart_id optional. "An order must have a shopper" makes orders.user_id required.
Many-to-manyCannot be stored in two tables. Resolve it with an associative table: cart_items, order_items.
The associative table earns its keepquantity, added_ts, line_no, unit_price. If it has nothing of its own, it is only plumbing.
Header and lineTwo grains, two tables. 985 orders, 1,434 lines - a single quantity column could only hold one line's worth.
Natural keyOnly when the business owns and keeps the identifier stable. M07 qualifies; an email address never does.
Surrogate keyAn integer the system owns. Postgres: GENERATED ALWAYS AS IDENTITY. Many writers: UUIDv7, so ids stay sortable.
Composite keyThe grain sentence the database enforces. Read it as "one row per ...". Next: b4, the physical layer.