learn-data-modeling-with-phoebe / Builder session 4 of 10
Learn Data Modeling with Phoebe · Builder track · Session 4 of 10

Physical OLTP

The logical model becomes DDL. Types, NOT NULL, CHECK, foreign keys, and the first few indexes - each one a rule you are moving out of somebody's head and into the database, where it gets enforced at 3am without being asked. This session ships the file the whole track runs on, plus the generator that fills it with 90 days of deterministic marketplace data.

🟡 Builder track Analysts · analytics engineers · DE · DS Runs in your browser · real SQLite 45 min
0-3 · Welcome 3-18 · Types, constraints, indexes 18-42 · Build-along: write the DDL, break it, enforce it 42-45 · Q&A
Part 0

Where the rules finally live

Sessions b1 to b3 produced a logical model: entities, attributes, cardinality, keys. None of it is enforced yet. Today every one of those decisions becomes a line of DDL, and the difference matters more than it looks: a rule in a document is a hope, a rule in the application is a hope with a deployment schedule, and a rule in the schema is a fact. A backfill script, a manual fix and a second service all have to obey it.

Live - presented in session Self-study - read after class ▶ Live SQL - editable & runnable Sources covered
★ What you walk out with today Bazaar's real schema shipped as sql/01_oltp_ddl.sql, the deterministic generator in python/gen_bazaar_data.py, and the habit of asking of every column: what type, what is forbidden, and what is enforcing it.
Part 1 · covers ANSI SQL, PostgreSQL and SQLite documentation

Types carry meaning, not just storage 6 min live

A type is the first constraint. It decides what can be stored, what comparisons mean, and how badly things go wrong at scale. Three choices matter more than all the others in a transactional schema: money, time, and identifiers.

Money In production: NUMERIC(12,2) Exact decimal arithmetic. Never a binary float: 0.10 is not representable, so a ledger drifts by cents. Time TIMESTAMP WITH TIME ZONE storing UTC, always. Naming carries the grain: order_ts is a moment, joined_date is a day. Identifiers GENERATED AS IDENTITY for a single writer. UUIDv7 when distributed: still sortable by time, unlike UUIDv4. This course stores money as REAL and time as ISO-8601 text, because SQLite in a browser has no decimal or timestamp type. That is a teaching compromise, written down in the DDL itself - which is what an honest model does with every compromise it makes.
🔍 Click to zoom - the three type decisions that cost the most when rushed
LiveNaming a column is a typing decision3 min

Bazaar uses two suffixes consistently, and the consistency is the point: *_ts is a moment in time (order_ts, created_ts, txn_ts), *_date is a calendar day (joined_date, launched_date). Anybody reading a query knows whether comparing to '2026-06-10' is going to behave.

  • Booleans say what they mean: is_guest, not flag. A boolean named after its meaning can be summed into a numerator later, which session b7 relies on.
  • Amounts say which amount: list_price on the product, unit_price on the order line, discount_amt beside it. Session b2 argued these are different facts; the naming is what stops them being treated as one.
  • Signed amounts say so in a comment: transactions.amount is negative for refunds. A sign convention nobody documented is a bug waiting for a SUM.
The 3am test If someone woken at 3am has to guess what a column holds, the name is wrong. This is the same discipline session a5 turns into an organisational standard, and the same discipline that makes the model queryable by an AI agent in b9.
Self-studyReading Bazaar's DDL as a set of decisions3 min read

Open sql/01_oltp_ddl.sql in the repo. Every table carries a comment saying which session owns the decision above it, so the file doubles as a map of the track. Three worth reading closely:

  • orders stores no amount. An order total is derivable from its lines, and a stored total that disagrees with its lines is the most common ecommerce data bug there is. Derive it, or store it as an explicitly reconciled snapshot - never as a casually maintained copy.
  • order_items carries merchant_id, which products already knows. That is point-in-time capture, not redundancy: a product can be transferred between merchants, and the sale belongs to whoever sold it.
  • transactions.decline_reason is nullable, and it is the only nullable column that earns it: the reason genuinely does not exist for an approved payment. Every other absence in this schema is either forbidden or given a real value.
Part 2 · the cheapest data quality you can buy

Four constraints, four bad rows stopped 6 min live

Constraints are data quality that costs nothing to run and cannot be forgotten. Each one stops a specific bad row at the door instead of surfacing it as a dashboard mystery three months later.

ConstraintBazaar exampleThe bad row it stops
NOT NULLusers.email, orders.order_tsan order that happened at no particular time, which silently drops out of every date filter
UNIQUEusers.email, products.sku, transactions.psp_refthe same payment recorded twice, which double-counts revenue
CHECKquantity > 0, list_price > 0, tier IN (...), status IN (...)a negative quantity, a free product, and a status nobody's code handles
FOREIGN KEYorder_items.product_idproductsa sale of a product that does not exist, which becomes an orphan row and a missing row in every report
LiveA declared constraint is not an enforced one4 min

This is the trap that catches experienced teams. SQLite parses and stores REFERENCES clauses but does not enforce them unless foreign keys are switched on for the connection. A schema can be full of relationships that enforce nothing, and the only symptom is orphan rows appearing months later.

PRAGMA foreign_keys = OFF;

CREATE TABLE merchants (merchant_id TEXT PRIMARY KEY, merchant_name TEXT NOT NULL);
CREATE TABLE products (
  product_id  INTEGER PRIMARY KEY,
  merchant_id TEXT NOT NULL REFERENCES merchants(merchant_id),
  sku         TEXT NOT NULL UNIQUE,
  list_price  REAL NOT NULL CHECK (list_price > 0)
);
INSERT INTO merchants VALUES ('M07', 'Nimbus Audio');

-- M99 does not exist. With foreign keys off, this is accepted.
INSERT INTO products VALUES (1, 'M99', 'M99-1', 129.00);

SELECT p.product_id, p.merchant_id, m.merchant_name AS resolves_to
FROM products p
LEFT JOIN merchants m ON m.merchant_id = p.merchant_id;

The resolves_to column comes back NULL: an orphan. Now switch enforcement on and watch the same INSERT get rejected.

PRAGMA foreign_keys = ON;

CREATE TABLE merchants (merchant_id TEXT PRIMARY KEY, merchant_name TEXT NOT NULL);
CREATE TABLE products (
  product_id  INTEGER PRIMARY KEY,
  merchant_id TEXT NOT NULL REFERENCES merchants(merchant_id),
  sku         TEXT NOT NULL UNIQUE,
  list_price  REAL NOT NULL CHECK (list_price > 0)
);
INSERT INTO merchants VALUES ('M07', 'Nimbus Audio');

-- Same statement, now rejected. Comment it out to see the rest run.
INSERT INTO products VALUES (1, 'M99', 'M99-1', 129.00);

SELECT 'never reached' AS note;
Check yours today In SQLite, PRAGMA foreign_keys; returns 0 or 1. In Postgres, constraints are always enforced but can be declared NOT VALID, so query pg_constraint.convalidated. Either way, the question is the same: is this rule enforced, or just written down?
Self-studyWhat a CHECK cannot do3 min read

A CHECK constraint sees one row of one table. That makes it perfect for quantity > 0 and useless for anything cross-row or cross-table:

Rule needs only one row? Use a CHECK constraint External test needed yes no e.g. quantity > 0 is one row. Return <= quantity sold spans two rows - a CHECK cannot see both.
🔍 Click to zoom - a CHECK sees one row; everything cross-row needs a test
  • "A return cannot exceed the quantity sold" needs the order line as well as the return. A CHECK cannot express it, so the rule has to live in application logic, a trigger, or a validation job - and wherever you put it, write down that you put it there. The capstone in b10 makes you make this exact call.
  • "An order's total equals the sum of its lines" is cross-row. This is precisely why orders stores no total: the safest way to enforce an invariant is to remove the ability to violate it.
  • "Only one current version per merchant" is cross-row too, which is why session b6's slowly-changing dimension gets a validation check in python/validate_model.py rather than a constraint.

The pattern: constraints for what a single row can prove, tests for everything else. A model with no tests is trusting the parts of itself that constraints cannot reach.

Demo 1 of 2

Write a table properly, then try to put bad data in it ★ 12 min · everyone builds

Build Bazaar's order_items from scratch - composite primary key, CHECK constraints, two foreign keys - then attack it. Every rejection you get is a bad row that will never reach a dashboard.

Create the parents and the line table, with the composite key (order_id, line_no) that says what one row is.

Insert two good lines. They go in without comment, which is what correct data looks like.

Attack it. Uncomment one violation at a time and read the error. Each error message names the rule that saved you.

PRAGMA foreign_keys = ON;

CREATE TABLE orders (
  order_id  INTEGER PRIMARY KEY,
  order_ts  TEXT NOT NULL,
  status    TEXT NOT NULL
);
CREATE TABLE products (
  product_id INTEGER PRIMARY KEY,
  sku        TEXT NOT NULL UNIQUE,
  list_price REAL NOT NULL CHECK (list_price > 0)
);
CREATE TABLE order_items (
  order_id     INTEGER NOT NULL REFERENCES orders(order_id),
  line_no      INTEGER NOT NULL,
  product_id   INTEGER NOT NULL REFERENCES products(product_id),
  quantity     INTEGER NOT NULL CHECK (quantity > 0),
  unit_price   REAL    NOT NULL CHECK (unit_price >= 0),
  discount_amt REAL    NOT NULL DEFAULT 0 CHECK (discount_amt >= 0),
  PRIMARY KEY (order_id, line_no)
);

INSERT INTO orders   VALUES (9001, '2026-06-01 12:04:11', 'paid');
INSERT INTO products VALUES (1, 'M07-1', 189.00), (2, 'M07-2', 129.00);

INSERT INTO order_items VALUES (9001, 1, 1, 1, 189.00, 0);
INSERT INTO order_items VALUES (9001, 2, 2, 2, 129.00, 12.90);

-- INSERT INTO order_items VALUES (9001, 2, 1, 1, 189.00, 0);   -- duplicate (order_id, line_no)
-- INSERT INTO order_items VALUES (9001, 3, 1, 0, 189.00, 0);   -- quantity must be > 0
-- INSERT INTO order_items VALUES (9001, 4, 42, 1, 189.00, 0);  -- product 42 does not exist
-- INSERT INTO order_items VALUES (9002, 1, 1, 1, 189.00, 0);   -- order 9002 does not exist

SELECT order_id, line_no, quantity, unit_price, discount_amt,
       ROUND(quantity * unit_price - discount_amt, 2) AS net_amount
FROM order_items
ORDER BY order_id, line_no;
Real world

Every constraint here was once an incident somewhere. The composite key exists because a retry once wrote the same line twice and revenue was 4% high for a quarter. The quantity > 0 check exists because a returns process wrote negative quantities into sales and nobody could work out why one merchant appeared to be paying customers. Constraints are institutional memory that survives the people who learned it.

✗ No composite key a retry writes the same line twice nothing stops the duplicate row revenue reads 4% high for a quarter ✓ Composite key enforced (order_id, line_no) is the key the duplicate retry is rejected revenue matches the source exactly A retry once duplicated a line and overstated revenue 4% for a quarter - the composite key stops it.
🔍 Click to zoom - a retry without a composite key once cost a quarter of revenue
Demo 2 of 2

Your turn: read the physical layer of the real schema ★ 10 min · build your own

These run against Bazaar's actual source database - the one sql/01_oltp_ddl.sql and the generator produced.

LiveQ1 · List the indexes, and ask why each one exists3 min
SELECT name AS index_name, tbl_name AS on_table
FROM sqlite_master
WHERE type = 'index' AND name NOT LIKE 'sqlite_%'
ORDER BY tbl_name, name;

Read them as questions the application asks: orders by time, orders by user, lines by product, lines by merchant, payments by order, payments by time, cart items by product. Nothing indexed speculatively. Analytical access paths get solved by the star schema in b6 and b7, not by adding indexes here.

LiveQ2 · Prove the NOT NULL and UNIQUE promises hold4 min
SELECT 'users with no email'          AS check_name, COUNT(*) AS offending_rows FROM users        WHERE email IS NULL
UNION ALL SELECT 'orders with no timestamp',       COUNT(*) FROM orders       WHERE order_ts IS NULL
UNION ALL SELECT 'duplicate emails',               COUNT(*) - COUNT(DISTINCT email) FROM users
UNION ALL SELECT 'duplicate SKUs',                 COUNT(*) - COUNT(DISTINCT sku) FROM products
UNION ALL SELECT 'duplicate payment references',   COUNT(*) - COUNT(DISTINCT psp_ref) FROM transactions
UNION ALL SELECT 'non-positive quantities',        COUNT(*) FROM order_items  WHERE quantity <= 0
UNION ALL SELECT 'decline reason on an approval',  COUNT(*) FROM transactions WHERE status = 'approved' AND decline_reason IS NOT NULL;

This is the shape of every data test worth writing: name the promise, count the rows that break it, expect zero. python/validate_model.py is 27 of these, run over the analytical model, and it exits non-zero when any of them fails.

Self-studyQ3 · Why the generator is deterministic3 min

No SQL. python/gen_bazaar_data.py seeds its random generator with 42 and never calls the clock. That single decision is what makes every number on every page of this course reproducible: 985 orders, 1,434 lines, 111,906.48 SGD of settled net revenue, the same on your laptop as in the browser lab.

It also makes the four buried stories teachable. A demand shock on 2026-06-18, a checkout incident on 2026-06-10, one merchant's collapse and a stockout are all deliberate, and they land on the same dates every time you regenerate - so session b8 can ask you to find them and then tell you whether you were right.

The habit to steal: test data with a fixed seed, a documented story, and no wall-clock dependency. Random test data that changes every run cannot be reasoned about, and a model you cannot reason about is a model you cannot trust.

Homework

Try it yourself - this week ◐ 20-30 min total

Source material

Sources covered

Full source map in materials/official-course-map.md. This page covers:

ANSI SQL with the PostgreSQL and SQLite documentation - constraint semantics, type selection, PRAGMA foreign_keysParts 1-2 + both demos · declared versus enforced
Codd / relational fundamentals - keys and integrity as schema-level guaranteesPart 2 · composite keys and referential integrity in DDL
Identifier strategy at scale - IDENTITY columns, UUIDv7 sortabilityPart 1 · named with the trade-off; distributed writers are out of scope
Index and query tuningDemo 2 Q1 · access-path indexes only; execution plans and partitioning are out of scope by design
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · A SQLite schema declares REFERENCES products(product_id) on every line table, and orphan rows keep appearing. Why?

SQLite parses and stores the clause but enforces it only per connection, so a schema full of relationships can enforce nothing. A constraint you declared but never enabled is a comment. Check it before you trust it.

2 · Why does Bazaar's orders table store no order total?

A total that disagrees with its lines is the most common ecommerce data bug there is, and no CHECK constraint can catch it because the rule is cross-row. Derive it, or store it as an explicitly reconciled snapshot - never as a casually maintained copy.

3 · Which rule can a CHECK constraint NOT enforce?

A CHECK sees one row of one table. Single-row rules belong in constraints; cross-row and cross-table rules belong in application logic, a trigger, or a validation test - and wherever you put them, document where. That is the call the b10 capstone makes you make.

Builder session 4 cheat sheet · pin this

MoneyNUMERIC/DECIMAL in production. Never binary floats - 0.10 is not representable and ledgers drift.
TimeTIMESTAMP WITH TIME ZONE, store UTC. Name it *_ts for a moment, *_date for a day.
IdentifiersIDENTITY for one writer, UUIDv7 when distributed so ids stay sortable.
Four constraintsNOT NULL, UNIQUE, CHECK, FOREIGN KEY. Each stops one bad row at the door.
Declared ≠ enforcedPRAGMA foreign_keys = ON in SQLite; NOT VALID constraints in Postgres. Verify, do not assume.
CHECK sees one rowCross-row and cross-table rules need logic or tests. Write down where you put them.
Index the access pathsOnly what the app queries. Every index taxes writes; analytics gets a star, not more indexes.
Shipssql/01_oltp_ddl.sql + python/gen_bazaar_data.py (seed 42, reproducible). Next: b5, why analysts cannot query this.