learn-data-modeling-with-phoebe / Builder session 7 of 10
Learn Data Modeling with Phoebe · Builder track · Session 7 of 10

Fact design

Session b6 built the dimensions that answer "by what". This session builds the tables that answer "how much" - and the whole discipline comes down to one sentence you have to write before you write any DDL. Bazaar gets three facts at three grains: what was sold, what was paid, what was wanted. You will see, in real SQL, what happens when someone joins two of them, and why the resulting number is worse than an error.

🟠 Builder track Analysts · analytics engineers · DE · DS Hands-on · real SQLite 45 min
0-3 · Welcome 3-16 · Grain first, then three facts 16-42 · Build-along: the double count, and the fix 42-45 · Q&A
Part 0

One sentence decides the whole table

A fact table is where the numbers live, so it is the table people fight about. Almost every one of those fights traces back to the same missing artifact: nobody wrote down what one row means. This session makes that sentence the first deliverable, not a comment somebody adds later, and then builds Bazaar's three facts around it.

Two files ship today, and they are the spine of the whole star: sql/11_fact_ddl.sql is the DDL for the three facts, and sql/12_oltp_to_star.sql is the load that fills them from the source tables you designed in b4. Read the comment block at the top of sql/11 before class if you want a head start - it is this session in 30 lines.

Live - presented in session Self-study - read after class ▶ Live SQL - editable & runnable Sources covered
★ What you walk out with today The habit of writing the grain sentence before the DDL, Bazaar's three fact tables and the reason there are three, a working test for whether a measure can be summed, and a query you have run yourself that inflates revenue by joining two facts - so you never write it again by accident.
Part 1 · covers Kimball ch.3 - grain and fact table types

Write the grain sentence first 7 min live

The grain of a fact table is what one row means, stated as one sentence, in business words, before any column exists. It is not documentation. It is the design decision every other decision hangs off: which dimensions can attach, which measures are legal, and what COUNT(*) is allowed to be called. Bazaar's three facts each get one sentence, and none of the three sentences needs the word "and".

fact_order_item GRAIN: one product line on one order. 1,434 rows Answers: what was SOLD Measures: net_amount, margin_amount, quantity Degenerate: order_id, line_no fact_transaction GRAIN: one payment attempt. 1,060 rows Answers: what was PAID Measures: approved_amount, declined_amount, is_declined Degenerate: txn_id, order_id fact_cart GRAIN: one cart. 2,229 rows Answers: what was WANTED Measures: cart_value, abandoned_value, converted_flag Degenerate: cart_id, order_id Three sentences, so three tables. One order can carry 3 product lines and 2 payment attempts, and those two numbers do not divide into each other. Force all three facts into one table and you get 6 rows per order, revenue counted twice, and no honest answer to "what is one row". If the sentence needs "and", you have two facts.
🔍 Click to zoom - three grains, three tables, three different questions
LiveWhat it means if you cannot write the sentence3 min

Try it out loud on any table you are about to create. "One row is one..." If you get stuck, you have learned something important, and it is always one of three things:

  • You need the word "and". "One row is one order and its payment" is two grains wearing one name. That table will double-count the moment an order gets a second attempt, which happens on the first day in production.
  • You need the word "sometimes". "One row is one order, or sometimes a refund line" means the table's meaning depends on the row, so no measure on it is safe to sum. Split it.
  • You can only describe the query. "One row is what the weekly revenue report needs" is not a grain, it is a screenshot. You do not have a fact table - you have a query result you are about to freeze, and it will be wrong the first time somebody asks a slightly different question. This is the most common way a warehouse fills up with tables nobody can reuse.
The sentence is a contract, so put it where it cannot be lost In sql/11_fact_ddl.sql every table opens with -- GRAIN: ..., and the sentence is then enforced: UNIQUE (order_id, line_no) on fact_order_item is the grain written as a constraint. A grain in a wiki drifts. A grain in a UNIQUE index gets defended by the database.
LiveWhy Bazaar has three facts and not one4 min

The pressure to build one big table is real and it always sounds reasonable: "analysts want one place to go". Here is the arithmetic that settles it. Take a single Bazaar order with 3 product lines and 2 payment attempts, and put both facts in one table:

  • fact_order_item would contribute 3 rows, one per line. That is what was sold.
  • fact_transaction would contribute 2 rows, one per attempt. That is what was paid.
  • Together, at one grain, you get 3 x 2 = 6 rows, because there is no rule that pairs line 2 with attempt 1. Every line's revenue now appears twice, so revenue for that order is exactly 2x the truth.

Change the order to 4 lines and 3 attempts and the multiplier becomes 3x. The error is not a constant you could correct for; it varies row by row with how many times a customer's card was retried. That is why the answer is never "divide by the number of attempts".

And the third fact earns its place for a different reason. fact_cart holds demand that never became revenue - 2,229 carts against 1,434 sold lines. Delete it and "sales dropped" and "fewer people wanted to buy" become the same sentence, which is exactly the distinction session b8 needs to make.

Self-studyThe three fact types, and the one with no measures at all4 min read

Kimball's three fact types are really three different answers to "when does a row appear":

  • Transaction fact - a row appears when something happens, and never changes again. All three of Bazaar's facts are this type, which is the type you should default to. Insert-only tables are the easiest thing in a warehouse to load, test and trust.
  • Periodic snapshot - a row appears per entity per period whether anything happened or not: inventory on hand each night, account balance each month-end. The measures are usually semi-additive, and the table grows on a schedule rather than with the business.
  • Accumulating snapshot - one row per thing that moves through a pipeline, updated in place as it hits each milestone. An order with placed_date, packed_date, shipped_date, delivered_date is the classic. It is the only fact type you routinely UPDATE, which is why it is also the one that needs the most care.

Factless facts are the fourth shape, and the one people are slowest to reach for: a fact table with dimension keys and no measures, recording that a combination of things occurred. A row per product viewed by a user, a row per promotion a merchant was eligible for. You count rows and you count absences - "which products were eligible for the sale and never sold" is a question only a factless fact can answer.

Bazaar has a near-miss worth noticing. fact_cart has measures, so it is a normal transaction fact, but a fact_cart_item at the grain "one product in one cart" would be almost factless: quantity, and nothing else. The capstone in b10 asks you to decide whether returns need one.

Part 2 · covers Kimball - additivity, degenerate dimensions, conformance

Measures, degenerates, and how facts meet 6 min live

Once the grain is fixed, two questions remain: which of these columns is safe to add up, and what happens when a question spans two facts. The first has a three-way answer you can test mechanically. The second has exactly one legal answer, and the diagram below is what happens when someone finds an illegal one.

fact_order_item · 3 lines line_no = 1 line_no = 2 line_no = 3 x fact_transaction · 2 attempts attempt 1 attempt 2 = JOIN ON order_id · 6 rows line 1 x att 1 line 1 x att 2 line 2 x att 1 line 2 x att 2 line 3 x att 1 line 3 x att 2 Every line is now paired with every attempt, so every line's revenue is summed twice. Across all of Bazaar this join turns 111,906.48 of settled revenue into 121,003 - about 8% too high, produced by a query that ran with no error, no warning and no NULL. Silently missing or silently inflated money is the worst failure mode in this course. The fix is never a cleverer join. Facts meet each other in exactly two legal ways: through a conformed dimension both of them share, or by aggregating each one to a common grain FIRST and joining those results. Never fact to fact.
🔍 Click to zoom - the fan-out, and the only two legal ways two facts can meet
Column on a factClassThe rule
net_amount, margin_amount, quantityadditiveSafe to SUM across every dimension. This is what a fact table is for, and most columns should be here.
approved_amount, declined_amountadditiveDeliberately pre-split from one signed amount so that both halves are summable without a CASE expression.
abandoned_value, converted_flagadditiveA flag stored as 0/1 is a numerator. SUM it to count, and it stays additive across every dimension.
stock on hand, account balance (not in Bazaar)semi-additiveSum across products or accounts, never across time. Ten units on Monday plus ten on Tuesday is not twenty.
unit_pricenon-additiveA rate, kept for reference only. Never SUM it. Recompute from gross_amount / quantity if you need it.
AOV, decline rate, conversion rate, margin %non-additiveNot stored at all. A ratio lives as its numerator and denominator, and the division happens at read time.
order_id, txn_id, cart_id, line_nodegenerateNot a measure and not a dimension - an identifier with no attributes worth a table of its own. Kept on the fact for COUNT(DISTINCT) and for drill-down to the source row.
decline_reasonjudgement callLow-cardinality text left on the fact rather than given a dimension. Defensible while it has no attributes; the moment somebody wants "is this reason retryable", it earns a table.
LiveThe additivity test, and why a stored ratio is a trap3 min

The test is mechanical: can I SUM this column across every dimension on the table and get a number a business person would accept? Run it column by column. Anything that fails is a ratio, a rate or a snapshot in disguise.

  • Additive passes everywhere. SUM(net_amount) by merchant, by day, by product, by hour, all at once - all valid, all reconcilable to the same total.
  • Semi-additive passes across some dimensions and fails across time. There is no semi-additive measure in Bazaar, which is lucky, because they are the ones that produce plausible nonsense: a year-end inventory figure that is twelve months of stock added together.
  • Non-additive fails immediately, and the giveaway is always division. If a column was computed with a /, storing it lets someone average it later, and an average of averages is wrong by an amount nobody can predict.

So the rule in sql/11 is absolute: a ratio is stored as its numerator and its denominator, never as the ratio. The marts in b8 follow it too - mart_merchant_daily ships aov_numerator and aov_denominator rather than aov, precisely so that the AOV of a week is computable from the AOV of its days.

The number that proves it Bazaar's AOV is 116.93: settled revenue divided by settled orders. Average the line values instead and you get 80.68. Both are one line of SQL, both run, and only one is AOV. Storing the ratio is what makes the wrong one available.
LiveLEFT JOIN plus COALESCE, and never an inner join3 min

Every dimension key in sql/12_oltp_to_star.sql is resolved the same way, and it looks needlessly defensive until the first time it saves you:

  • LEFT JOIN the dimension, so a lookup that misses returns NULL instead of removing the row.
  • COALESCE(dim.key, -1), so that NULL becomes the unknown member session b6 put in every dimension.
  • Never JOIN. An inner join drops the fact row when the lookup misses. The load succeeds, the row counts look plausible, and the money on that row is simply gone.

Think about which failure you would rather debug. With the unknown member, a merchant that failed to load shows up as an "unknown" row on every report, and somebody asks about it within a day. With an inner join, revenue is quietly lower than it should be, everybody assumes the market softened, and the reconciliation happens at quarter end if it happens at all.

This is also why validate_model.py asserts an unknown-member rate of 0.0% on all three facts as one of its 27 checks. A rate of zero is not the point - a rate you are measuring is the point.

Self-studyConformed dimensions: how facts are supposed to meet3 min read

A dimension is conformed when two or more facts use the identical dimension table, with the same keys and the same attribute meanings. Bazaar's dim_date, dim_time_of_day and dim_user are shared by all three facts; dim_merchant is shared by sales and carts.

Conformance is what makes "drilling across" possible, and drilling across is the legal way to answer a question that spans facts:

  • Aggregate each fact separately to the shared grain. Revenue by date from fact_order_item, attempts by date from fact_transaction, carts by date from fact_cart.
  • Then join the results on the conformed key. Three separate aggregations, one join, no fan-out possible, because each side already has exactly one row per date.

mart_funnel_daily in session b8 is that pattern written out: three CTEs, one per fact, each aggregated to one row per day, then joined. It is the single most reusable shape in dimensional modeling, and once you have seen it the temptation to join two facts directly disappears.

The failure mode on the other side is worth naming too. If sales says "merchant" and carts says "seller", and the two tables list slightly different merchants, then no amount of correct SQL will make the two facts comparable. Conformance is a naming and ownership discipline before it is a technical one, which is why session a5 treats it as a standard rather than a schema detail.

Part 3 · your live star

The three facts, in your browser 3 min live

Every editor below runs against the full Bazaar star, marts and agent views, rebuilt from source in your tab before each run. Nothing you type can break the next example. Start by reading the three grains off the database instead of off a slide.

LiveThree facts, three row counts, three grain sentences3 min

Press ▶ Run. The first run downloads the engine once, then caches it.

SELECT 'fact_order_item'  AS fact,
       'one product line on one order' AS grain_sentence,
       COUNT(*)                        AS rows,
       COUNT(DISTINCT order_id)        AS distinct_orders
FROM fact_order_item
UNION ALL
SELECT 'fact_transaction', 'one payment attempt',
       COUNT(*), COUNT(DISTINCT order_id) FROM fact_transaction
UNION ALL
SELECT 'fact_cart', 'one cart',
       COUNT(*), COUNT(DISTINCT order_id) FROM fact_cart;
Read the third column as a warning label All three facts carry order_id, and all three have a different number of rows per order. That is precisely why order_id is a degenerate dimension and not a join path between facts: it identifies the order, it does not make the tables compatible.
Demo 1 of 2

Commit the double count, then fix it ★ 13 min · everyone builds

You will write the wrong query on purpose. It is the single most valuable thing in this session, because the wrong query looks completely reasonable and produces a number that is too plausible to question.

Find the fan-out on one order: 3 sales lines, 2 payment attempts, so 6 rows and every amount stored twice.

Scale it up to the whole marketplace. Watch settled revenue go from 111,906.48 to 121,003 with no error message.

Fix it properly by aggregating each fact to the order grain before joining, and confirm the total lands back on 111,906.48.

SELECT s.order_id, s.line_no, s.product_name, s.net_revenue,
       p.txn_id, p.is_approved
FROM v_sales_line s
JOIN v_payment_attempt p ON p.order_id = s.order_id
WHERE s.order_id = 25
ORDER BY s.line_no, p.txn_id;

-- the same order, summed both ways
SELECT (SELECT ROUND(SUM(net_revenue), 2) FROM v_sales_line
          WHERE order_id = 25)                       AS truth,
       (SELECT ROUND(SUM(s.net_revenue), 2)
          FROM v_sales_line s
          JOIN v_payment_attempt p ON p.order_id = s.order_id
         WHERE s.order_id = 25)                      AS after_the_join;
SELECT 'joined the two facts on order_id' AS how,
       COUNT(*)                          AS rows_after_join,
       ROUND(SUM(s.net_revenue), 2)      AS revenue_reported
FROM v_sales_line s
JOIN v_payment_attempt p ON p.order_id = s.order_id
WHERE s.is_paid = 1
UNION ALL
SELECT 'v_sales_line alone (the truth)', COUNT(*), ROUND(SUM(net_revenue), 2)
FROM v_sales_line
WHERE is_paid = 1;
WITH sales AS (          -- fact 1, aggregated to the order grain
  SELECT order_id, SUM(net_revenue) AS net_revenue
  FROM v_sales_line
  WHERE is_paid = 1
  GROUP BY order_id
),
pay AS (                 -- fact 2, aggregated to the SAME grain
  SELECT order_id, COUNT(*) AS attempts, SUM(is_declined) AS declines
  FROM v_payment_attempt
  GROUP BY order_id
)
SELECT COUNT(*)                        AS one_row_per_order,
       ROUND(SUM(s.net_revenue), 2)    AS net_revenue,
       SUM(p.attempts)                 AS attempts_on_those_orders,
       SUM(p.declines)                 AS declines_on_those_orders
FROM sales s
LEFT JOIN pay p ON p.order_id = s.order_id;
Real world

The bug that survives because the number is believable. A payments company reported gross processing volume by joining its settlements fact to its authorizations fact on a reference id. Retries are rare, so the inflation was around 6% - large enough to matter to a board, small enough that every month looked like a good month rather than a broken query. It was found eighteen months later by an accountant reconciling to the bank, not by an engineer. An error gets fixed the day it appears; a plausible number gets quoted in a strategy deck. Session b9 removes this footgun structurally, by never exposing both facts in one view.

Demo 2 of 2

Your turn: measures, degenerates and the unknown member ★ 10 min · build your own

Three short investigations, each one testing a rule from Part 2 against the real star. Edit them, break them, re-run. Every editor starts from a fresh database.

LiveQ1 · Additive, non-additive, and the ratio you must not store4 min
SELECT 'net_revenue' AS measure, 'additive' AS additivity,
       ROUND(SUM(net_revenue), 2) AS value_summed
FROM v_sales_line WHERE is_paid = 1
UNION ALL
SELECT 'margin', 'additive', ROUND(SUM(margin), 2)
FROM v_sales_line WHERE is_paid = 1;

SELECT ROUND(SUM(net_revenue), 2)                            AS numerator_revenue,
       COUNT(DISTINCT order_id)                              AS denominator_orders,
       ROUND(SUM(net_revenue) / COUNT(DISTINCT order_id), 2) AS aov_divided_at_read_time,
       ROUND(AVG(net_revenue), 2)                            AS aov_if_you_average_lines,
       ROUND(100.0 * SUM(margin) / SUM(net_revenue), 2)      AS margin_pct
FROM v_sales_line
WHERE is_paid = 1;

116.93 against 80.68, from the same rows. The gap exists because the denominator changed silently from orders to lines. Notice that margin_pct of 46.89% is computed the same way, from two sums - it is never a stored column, and that is why it is still correct when you group by merchant, by month or by both.

LiveQ2 · What a degenerate dimension is actually for3 min
SELECT COUNT(*)                 AS fact_rows,
       COUNT(DISTINCT order_id) AS distinct_orders,
       MAX(line_no)             AS most_lines_on_one_order
FROM fact_order_item;

-- the drill-down job: from a merchant total back to the individual orders
SELECT s.merchant_name, s.order_id,
       COUNT(*)                     AS lines_on_order,
       ROUND(SUM(s.net_revenue), 2) AS order_revenue
FROM v_sales_line s
WHERE s.is_paid = 1 AND s.merchant_name = 'Nimbus Audio'
GROUP BY s.merchant_name, s.order_id
ORDER BY order_revenue DESC
LIMIT 8;

1,434 rows and far fewer distinct orders is the whole reason COUNT(*) must never be called "orders". Session b9 shows an AI agent making exactly that mistake and reporting 481 orders for June instead of 327 - a query that runs, on a correct table, and is still wrong because nobody published the grain.

Self-studyQ3 · Break a dimension and watch the inner join hide the money4 min

This is the argument for LEFT JOIN plus COALESCE, made as an experiment. Delete one merchant from the dimension - a stand-in for a load that half failed - then read the same revenue two ways.

-- Simulate a dimension row that failed to load. Foreign keys are switched off
-- for this one demo on purpose: in a real warehouse the load would have
-- inserted the fact and skipped the dimension, which is exactly this state.
PRAGMA foreign_keys = OFF;
DELETE FROM dim_merchant WHERE merchant_id = 'M07';

SELECT 'INNER JOIN - rows silently dropped' AS how,
       COUNT(*)                      AS fact_rows_kept,
       ROUND(SUM(f.net_amount), 2)   AS revenue_reported
FROM fact_order_item f
JOIN dim_merchant m ON m.merchant_sk = f.merchant_sk
WHERE f.is_paid = 1
UNION ALL
SELECT 'LEFT JOIN - every row survives', COUNT(*), ROUND(SUM(f.net_amount), 2)
FROM fact_order_item f
LEFT JOIN dim_merchant m ON m.merchant_sk = f.merchant_sk
WHERE f.is_paid = 1;

-- and the check that would have caught it: unknown members per fact
SELECT 'fact_order_item' AS fact,
       SUM(CASE WHEN merchant_sk = -1 THEN 1 ELSE 0 END) AS unknown_merchant,
       SUM(CASE WHEN product_sk  = -1 THEN 1 ELSE 0 END) AS unknown_product,
       SUM(CASE WHEN user_sk     = -1 THEN 1 ELSE 0 END) AS unknown_user,
       SUM(CASE WHEN date_key    = -1 THEN 1 ELSE 0 END) AS unknown_date
FROM fact_order_item;

The LEFT JOIN still reports 111,906.48. The inner join reports less, and nothing anywhere says so. Now change the load in your head: instead of the dimension row being missing, imagine the fact arrived with a merchant id the dimension has never seen. COALESCE(..., -1) is what turns that into a visible "unknown" row on a report rather than a silent subtraction from revenue, and the last query is the one-line audit that turns it into a number somebody watches.

Homework

Try it yourself - this week ◐ 20-30 min total

Source material

Sources covered

Full source map in materials/official-course-map.md. This page covers:

Kimball & Ross, The Data Warehouse Toolkit - grain, fact table types, additivity, degenerate dimensions, factless factsParts 1-2 · the grain sentence as first deliverable, and the three-way additivity test
Kimball - conformed dimensions and drilling acrossPart 2 + Demo 1 · the only two legal ways two facts meet, demonstrated both wrong and right
ANSI SQL / SQLite - UNIQUE on the grain, CHECK constraints, indexes on fact keyssql/11 · the grain enforced as a constraint; constraint semantics proper are b4
Kimball - SCD type 2 keys as they arrive on the factsql/12 · is_current = 1 on the lookup; the dimension side is session b6
Warehouse load mechanics - incremental loads, MERGE, late-arriving factsOut of scope by design - sql/12 is a full refresh; loading is learn-data-warehouse
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · You are about to create a fact table, and the only way you can describe one row is "whatever the weekly revenue report needs". What do you actually have?

If you cannot state the grain in business words without referring to a report, the table has no independent meaning. It will answer exactly one question, no dimension can be safely attached to it, and it becomes another table nobody can reuse. Write the sentence first, then the DDL.

2 · Joining v_sales_line to v_payment_attempt on order_id reports 121,003 instead of 111,906.48. What happened?

This is fan-out. The multiplier is lines times attempts, so it varies from order to order and cannot be corrected for after the fact. Aggregate each fact to a common grain first, or reach the other fact through a conformed dimension. Never join fact to fact.

3 · Why does sql/12 resolve every dimension key with LEFT JOIN plus COALESCE(..., -1) rather than an inner join?

The unknown member turns a lookup failure into a visible "unknown" row that somebody asks about within a day. An inner join turns the same failure into slightly lower revenue that everyone attributes to the market. Silently missing money is the worst failure mode in this course, which is why the loader never uses an inner join.

Builder session 7 cheat sheet · pin this

Grain sentence first"One row is one ___." Write it before the DDL, then enforce it with a UNIQUE constraint.
Cannot write it?Then you have a query result you are about to freeze, not a fact table.
Bazaar's three factsorder_item (what was sold, 1,434) · transaction (what was paid, 1,060) · cart (what was wanted, 2,229).
Why three3 lines x 2 attempts = 6 rows. The multiplier varies per order, so it cannot be divided out.
AdditivityAdditive (SUM anywhere) · semi-additive (not across time) · non-additive (never SUM).
RatiosStore the numerator and the denominator. AOV is 116.93, or 80.68 if the denominator slips to lines.
Degenerate dimensionsorder_id, txn_id, cart_id, line_no. No dimension of their own, kept for COUNT(DISTINCT) and drill-down.
Never an inner joinLEFT JOIN + COALESCE to -1. Facts meet only through a conformed dimension. Next: b8, the four questions.