How this track works
Ten sessions, one running project: Daybreak's six OLTP tables (customers, products, orders, order items, subscriptions, events) play the source system, raw and messy - dates stored as text and all. Session by session you will stage it (b2), reshape it into a star schema (b3-b4), track history with slowly changing dimensions (b5), load it properly (b6), serve it to the business (b7), make it fast and cheap (b8), meet the lakehouse (b9), and assemble the whole warehouse end to end (b10). If you took learn-sql-with-phoebe, this is the same database - now you build what b10 only hinted at.
OLTP vs OLAP: two jobs, two shapes 7 min live
Daybreak's app database answers thousands of tiny questions a day: "insert this order", "update this address", "show Ava her subscription". Each touches one row. The founder's questions are the opposite shape: "revenue by month by channel, all time" touches every row but only three columns. One database serving both is the design mistake warehouses exist to fix.
LiveThe day Daybreak's dashboard froze the checkout3 min▶
Every data team has this origin story. Daybreak's version: marketing opened a "lifetime revenue by cohort" dashboard at 9am on launch day. The query scanned every order ever placed - on the same database processing live checkouts. Carts spun. Support lit up. Engineering's fix was not "optimize the query"; it was "get analytics off the app database". That sentence is the birth certificate of every data warehouse.
- OLTP (online transaction processing): the app's workload. Many small reads/writes, one row at a time, correctness per transaction is sacred.
- OLAP (online analytical processing): the analyst's workload. Few queries, each scanning millions of rows but touching few columns.
- The fix is a copy: extract data out of the app database, restructure it for scans, and point every dashboard at the copy. The copy is the warehouse.
Why "just add an index" fails. Indexes accelerate finding few rows. Analytics reads all rows. There is no index for "scan everything and add it up" - the table's physical layout itself has to change. That layout change is Part 2.
Self-studyWhere the copy lives: warehouse, mart, lake3 min read▶
Three cousins you will hear named together (IBM's course opens with them; a4 in the leader track compares them for executives):
- Data warehouse: the structured, modeled copy - cleaned, typed, shaped for analytics. This course.
- Data mart: a slice of the warehouse for one team (finance mart, marketing mart). Session b7 builds them.
- Data lake: cheap storage for raw files of any shape - and the lakehouse puts warehouse-style tables directly on lake files. Session b9 is that story, and you will query real Parquet files when you get there.
Why column storage wins analytics 8 min live
The single deepest idea in warehousing: change how bytes sit on disk. A row store keeps each record's fields together - perfect for "fetch order 1017, all of it". A column store keeps each column's values together - perfect for "sum line_amount across every order ever". Same data, opposite physical layouts, and the layout decides which questions are fast.
LiveMeet DuckDB - the warehouse in your browser3 min▶
DuckDB is a real columnar OLAP engine - the same breed as Snowflake or BigQuery, minus the cloud. It compiles to WebAssembly, which is why every editor on these pages runs an honest-to-goodness warehouse engine inside your browser tab. First Run downloads the engine once (~8 MB, then cached); after that everything is local and instant.
- The SQL you know transfers: SELECT, JOIN, GROUP BY, window functions - all identical to what you learned in the SQL course.
- What is new here: columnar layout, CREATE TABLE AS pipelines, MERGE, Parquet - warehouse muscles SQLite never had.
- Honest note: the first Run on each page needs network for the one-time engine download. Everything after runs offline.
SELECT count(*) AS orders,
min(order_date) AS first_order,
max(order_date) AS last_order
FROM orders;
LiveSpot the mess: why this source needs a warehouse3 min▶
Look closely at what the app database actually hands us. Run this and check the typeof column:
SELECT order_id, order_date,
typeof(order_date) AS stored_as
FROM orders
LIMIT 4;
Dates stored as text. Revenue split across two tables. Statuses like refunded mixed into every count. This is normal, healthy OLTP design - and exactly what analytics cannot live with. Session b2 builds the staging layer that fixes it; tonight we just feel the pain.
The warehouse you are about to build 3 min live
Here is the whole track on one strip. Every session adds one layer to Daybreak's warehouse, and each layer exists to answer a business question the raw source fumbles.
| Session | You build | Daybreak gets |
|---|---|---|
| b2 | staging layer | typed, trusted copies of the source |
| b3-b4 | star schema: facts + dimensions | questions answered in one obvious join |
| b5 | slowly changing dimensions | history that does not rewrite itself |
| b6 | loading patterns: incremental + MERGE | tonight's orders in tomorrow's warehouse, safely |
| b7 | marts, views, rollups | each team its own clean slice |
| b8 | performance + cost tuning | fast dashboards, small bills |
| b9 | lakehouse: Parquet + external tables | one foot in the modern open stack |
| b10 | capstone: the whole pipeline | source to staging to star to served answer |
Feel columnar speed on 300,000 rows ★ 12 min · everyone builds
Thirty-three orders prove nothing. So this playground generates 300,000 synthetic order lines (table big_orders) before your query runs - then lets a columnar engine loose on them. Watch the millisecond counter under each result.
Run the aggregate below as-is. Note the time: a full scan of 300k rows, grouped and summed, in your browser tab.
Add status to the GROUP BY (both lines). Still instant - one more column read, the rest untouched.
Now try SELECT * FROM big_orders LIMIT 5; - fetching whole rows is the one thing this layout does NOT love. On 300k rows it survives; on 300 billion you would pay for it.
Say the lesson out loud: warehouses scan columns, not rows. Ask for the columns you need, never SELECT *.
SELECT channel,
count(*) AS orders,
ROUND(SUM(quantity * unit_price), 0) AS revenue
FROM big_orders
GROUP BY channel
ORDER BY revenue DESC;
The bill version of this lesson. Cloud warehouses charge by data scanned. SELECT * on a wide table reads every column and bills you for all of them; naming three columns reads three. Analysts who learned on row stores drag the * habit into BigQuery and the invoice notices before they do. Session b8 turns this into a cost playbook.
Your turn: three founder questions ★ 10 min · build your own
Same rules as the SQL course: write it, run it, read the error, fix it, run again. All three run against the fresh 300k-row big_orders table.
LiveQ1 · Monthly revenue: which month was biggest?4 min▶
SELECT strftime(order_date, '%Y-%m') AS month,
ROUND(SUM(quantity * unit_price), 0) AS revenue
FROM big_orders
GROUP BY month
ORDER BY month;
LiveQ2 · What share of orders get refunded, by channel?3 min▶
SELECT channel,
count(*) AS orders,
ROUND(AVG(CASE WHEN status = 'refunded' THEN 1.0 ELSE 0 END) * 100, 2) AS refund_pct
FROM big_orders
GROUP BY channel;
Self-studyQ3 · Weekend vs weekday: when do people order?3 min▶
A taste of what dim_date will make trivial in b4 - for now, derive it by hand:
SELECT CASE WHEN dayofweek(order_date) IN (0, 6)
THEN 'weekend' ELSE 'weekday' END AS day_type,
count(*) AS orders
FROM big_orders
GROUP BY day_type;
Try it yourself - this week ◐ 20-30 min total
- Bookmark this page - the editors are your permanent warehouse scratchpad, 300k rows on demand.
- Write one OLTP-shaped question and one OLAP-shaped question about any system you use at work. Check: does the first touch one row, and the second touch one column?
- On
big_orders, find the single biggest revenue day. (GROUP BYorder_date, order, limit.) - Explain to a colleague in two sentences why the company dashboard should not query the production app database. If they push back, you have Part 1's diagram.
- Bring your slowest known dashboard or query to b2 - the staging layer is where its fix begins.
Official sources covered
This track teaches the working core of the major data-warehousing curricula, run on a live engine instead of slides. Certificates, graded labs, and videos stay on the official platforms. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · Marketing's dashboard keeps timing out and checkout slows to a crawl whenever it loads. The textbook fix is...
Indexes speed up finding few rows; analytics reads all rows. The workloads want opposite designs, so you separate them - the warehouse is that copy.
2 · Why does a column store make SUM(line_amount) over 300k rows so fast?
Columnar layout stores each column's values together, so an aggregate touches only the columns it names - plus similar values compress better and crunch in batches.
3 · Which question is OLAP-shaped?
OLAP questions scan many rows but few columns (all history, two fields). One-row lookups and writes are OLTP - the app database's home turf.