learn-data-warehouse-with-phoebe / Builder session 7 of 10
Learn Data Warehouse with Phoebe · Builder track · Session 7 of 10

Marts and serving

Daybreak's warehouse loads nightly and keeps history honestly - and nobody outside the data team has touched it yet. Tonight the warehouse starts earning its keep: you carve team-shaped marts out of the core star, pre-aggregate the queries dashboards hammer all day, and learn the quiet contract that makes people trust a number they did not compute themselves.

🟠 Builder track Practitioners: analysts · DE · DS · PMs Runs in your browser · DuckDB 45 min
0-3 · Recap 3-20 · Marts, rollups & serving 20-42 · Build-along: Daybreak's marts 42-45 · Q&A
Part 0

Where the build stands

The stack under you tonight: typed staging (b2), the star - fct_order_line plus dim_date, dim_customer, dim_product (b3-b4), Type 2 history (b5), and idempotent MERGE loads keeping it fresh (b6). That is a complete warehouse core. What it is not, yet, is a product anyone consumes. Marts and serving turn the core into answers people actually open.

Live - presented in session Self-study - read after class ▶ Live warehouse - editable & runnable Official sources covered
★ What you walk out with today Two working marts (finance and marketing) with business policy baked in, the views vs materialized views vs tables freshness-cost-speed triangle, ROLLUP as the one-query cube, and the serving mindset: a mart is a promise, not a query.
Part 1 · covers IBM DW Fundamentals M1 data marts, DLAI C4 M4 views

Marts: each team its own governed slice 7 min live

Give every team raw access to the core star and each will re-invent the rules: finance excludes refunds AND cancellations, marketing excludes only cancellations, ops forgets both - and Monday's exec meeting has three different revenue numbers. A data mart is the fix: a slice of the core, shaped and filtered for one team, with the business rules written once, inside the definition.

Core star fct_order_line dim_date · dim_customer dim_product · SCD history one truth, all grain mart_finance_monthly policy: completed orders only mart_marketing_channel policy: refund rate by channel mart_top_products policy: category rollup, no PII CFO dashboard board pack growth team campaign reviews merchandising reorder planning The refund rule lives in ONE place - the mart definition - not in forty analysts' heads.
🔍 Click to zoom - one core, three team-shaped slices, three very different consumers
LiveView, materialized view, or table? The triangle4 min

A mart is a definition; you still choose its physical form. Three options, trading freshness, cost, and speed:

  • View: a saved query, computed at read time. Always fresh, zero storage - and every open of the dashboard pays the full compute. Perfect at Daybreak's size; DuckDB gives you these natively.
  • Materialized view: results stored, refreshed on a schedule. Reads are instant; the price is staleness between refreshes plus refresh compute. Cloud engines (Snowflake, BigQuery, Redshift) offer scheduled materialization; the concept is what transfers.
  • Table: you materialize it yourself - a CTAS your b6 load rebuilds each night. Maximum control, and now freshness is your pipeline's problem.

Rule of thumb: start as a view. Promote to materialized when a dashboard hits it hundreds of times a day and the compute bill (or latency) says so. You will feel this exact promotion pressure in b8.

Self-studyMart boundaries are political, not just technical2 min read

A mart's WHERE clause is a policy decision someone must own. "Finance revenue excludes refunded and cancelled orders" sounds technical, but it decides whose Monday number looks better - so get the definition agreed with the team that consumes it, write it into the mart, and document it where the leader track's a3 audience can find it. The alternative - every analyst re-deriving the rule - is how a company ends up arguing about arithmetic in the boardroom.

Part 2 · covers IBM DW Fundamentals M2 cubes & rollups

Pre-aggregation: ROLLUP, the one-query cube 7 min live

Old-school OLAP tools sold "cubes": every subtotal at every level, pre-computed. Modern SQL keeps the useful part in one clause. GROUP BY ROLLUP (month, channel) returns the detail rows, the per-month subtotals, and the grand total - one query, one scan.

GROUP BY ROLLUP (month_name, channel) - one result, three altitudes month_name · channel · revenue Jan · web · 152.00 Jan · app · 96.00 Jan · NULL · 248.00 (January subtotal) Feb · web · 134.00 ... every month, both altitudes ... NULL · NULL · 1685.00 (grand total) How to read it Detail rows: both columns filled - the ordinary GROUP BY output. Subtotal rows: channel is NULL - meaning "all channels" for that month. Grand total: every rolled column NULL - the whole table in one row. Illustrative numbers - run the real query in the playground below. In a ROLLUP result, NULL means "all" - the column was rolled up, not missing.
🔍 Click to zoom - detail, subtotals, and grand total from a single scan
LiveRun the cube: revenue by month and channel4 min

Spot the three altitudes in the live result: rows with both columns filled (detail), rows where channel is NULL (month subtotal), and the one row where both are NULL (grand total - which lands first, since we sort by revenue).

SELECT d.month_name, f.channel,
       ROUND(SUM(f.line_revenue), 2) AS revenue
FROM fct_order_line f
JOIN dim_date d ON f.date_key = d.date_key
WHERE f.status = 'completed'
GROUP BY ROLLUP (d.month_name, f.channel)
ORDER BY revenue DESC
LIMIT 12;
Self-studyWhen to pre-aggregate at all2 min read

Pre-aggregation is a cache, and every cache is a freshness debt. The test is arithmetic: a KPI tile hit 1,000 times a day over the same day-old numbers should read from a materialized rollup - computing it 1,000 times is pure waste. An analyst's one-off exploration should hit the star directly - fresh, flexible, and the scan cost is paid once. Daybreak's 53 fact rows make everything instant; the decision pattern is what you are rehearsing, and b8 attaches real costs to it.

Part 3 · covers DLAI C4 M4 serving

Serving surfaces: a mart is a promise 5 min live

"Serving" is everything downstream of the mart: BI dashboards reading it live, scheduled extracts mailed to the ops team, reverse ETL pushing segments back into the CRM, feature tables feeding ML models. Four consumers, one dependency in common - they all break if the mart changes shape under them.

LiveThe semantic contract3 min

The moment one consumer you do not control reads your mart, its definition stops being your private query and becomes an interface. Treat it like one:

  • Columns stay. Renaming revenue to net_revenue is a breaking change; every downstream dashboard and sync dies quietly. Add columns freely; remove or rename only with a migration.
  • Definitions are documented. "revenue = completed order lines only, refunds excluded" lives next to the mart, in words a CFO can read. This is the exact artifact the leader track's a3 session teaches executives to demand.
  • Grain is stated. "One row per month" or "one row per channel" - say it, so nobody double-counts by joining two marts of different grain.
Real world

The renamed column that killed a quarter of dashboards. A data team "cleaned up" a mart's column names on a Friday. By Monday, 40 dashboards showed blanks, the CRM sync had silently skipped three days, and trust in the platform took a quarter to rebuild. The fix cost one hour; the lesson - marts are contracts - cost a reorg.

Self-studyThe four serving surfaces, one line each2 min read
  • BI dashboards: the default. Point the tool at the mart, never at raw staging - the mart is where policy lives.
  • Scheduled extracts: CSVs on a schedule are unglamorous and everywhere. Same rule: export the mart, not raw tables.
  • Reverse ETL: warehouse-computed answers (segments, LTV, churn risk) pushed back into operational tools like the CRM. The warehouse graduates from reporting to running the business.
  • ML features: models train on mart-grade tables too - and gain the same reproducibility from stated grain and stable columns.
Demo 1 of 2

Build the finance and marketing marts ★ 12 min · everyone builds

Two marts, two teams, two policies - each written exactly once. Finance counts only completed orders. Marketing keeps refunds visible, because refund rate by channel is their KPI. Same core star underneath both.

Policy first: the finance mart's WHERE clause IS the policy - status = 'completed'. Refunded and cancelled orders never reach the CFO's numbers, by definition, not by analyst discipline.

Grain second: one row per month. dim_date supplies year, month, and month_name so the mart sorts and labels itself.

Run box 1, then box 2. Marketing's mart makes the opposite policy call: refunds stay in, surfaced as a rate per channel - one team's noise is another team's KPI.

Both are views: zero storage, always fresh, recomputed per read. Say when you would promote each to a materialized table (b8 makes you pay for the answer).

CREATE VIEW mart_finance_monthly AS
SELECT d.year, d.month, d.month_name,
       count(DISTINCT f.order_id) AS orders,
       ROUND(SUM(f.line_revenue), 2) AS revenue
FROM fct_order_line f
JOIN dim_date d ON f.date_key = d.date_key
WHERE f.status = 'completed'
GROUP BY d.year, d.month, d.month_name;

SELECT * FROM mart_finance_monthly
ORDER BY year, month;
CREATE VIEW mart_marketing_channel AS
SELECT channel,
       count(DISTINCT order_id) AS orders,
       ROUND(SUM(CASE WHEN status = 'completed'
                      THEN line_revenue ELSE 0 END), 2) AS completed_revenue,
       ROUND(100.0 * count(DISTINCT CASE WHEN status = 'refunded'
                                         THEN order_id END)
             / count(DISTINCT order_id), 1) AS refund_order_pct
FROM fct_order_line
GROUP BY channel;

SELECT * FROM mart_marketing_channel
ORDER BY completed_revenue DESC;
Demo 2 of 2

Your turn: two marts and a boundary call ★ 10 min · build your own

Same rules as always: write it, run it, read the error, fix it. Each box carries a working answer - try blind first.

LiveQ1 · The subscriptions mart: active vs cancelled, per plan4 min

The retention team wants one row per plan: how many subscriptions ever started, how many are still active, how many cancelled. Hint: cancel_date IS NULL means still active.

CREATE VIEW mart_subscriptions_by_plan AS
SELECT c.plan,
       count(*) AS subscriptions,
       SUM(CASE WHEN s.cancel_date IS NULL THEN 1 ELSE 0 END) AS active,
       SUM(CASE WHEN s.cancel_date IS NOT NULL THEN 1 ELSE 0 END) AS cancelled
FROM stg_subscriptions s
JOIN dim_customer c ON s.customer_id = c.customer_id
GROUP BY c.plan;

SELECT * FROM mart_subscriptions_by_plan
ORDER BY subscriptions DESC;
LiveQ2 · The top-products mart, with category rollup4 min

Merchandising wants product revenue with category subtotals and a grand total - Part 2's ROLLUP, now wearing a mart's name. Completed orders only (merchandising follows finance's policy here).

CREATE VIEW mart_top_products AS
SELECT p.category, p.name AS product,
       ROUND(SUM(f.line_revenue), 2) AS revenue
FROM fct_order_line f
JOIN dim_product p ON f.product_key = p.product_key
WHERE f.status = 'completed'
GROUP BY ROLLUP (p.category, p.name);

SELECT category, product, revenue
FROM mart_top_products
ORDER BY revenue DESC
LIMIT 10;
Self-studyQ3 · Which column would you NEVER put in a wide-access mart?3 min

Look back at dim_customer: customer_key, customer_id, name, city, country, plan, signup_date. The marketing mart is readable by half the company. Which columns stay out?

  • name - direct PII. A channel-performance mart has zero analytical need for it, so exposure is pure risk. Aggregate marts should carry counts of people, never people.
  • city - quasi-identifier. Alone it seems harmless; joined with plan and signup week it can single someone out in a small market. Keep country, drop city, unless the mart's purpose demands it and access is restricted.
  • The principle: marts minimize, by design. Every column in a mart is a disclosure decision - the leader track's governance session (a3) gives your executives the same rule from their side.
Homework

Try it yourself - this week ◐ 20-30 min total

Source material

Official sources covered

This track teaches the working core of the major data-warehousing curricula, run on a live engine instead of slides. Certificates, graded labs, and videos stay on the official platforms. This page covers:

IBM Data Warehouse Fundamentals (Coursera) - M1: data martsPart 1 + Demo 1 · marts as governed team slices, built live
IBM Data Warehouse Fundamentals (Coursera) - M2: cubes, rollups, materialized viewsParts 1-2 · ROLLUP as the SQL cube; the matview triangle
DeepLearning.AI Data Engineering C4 (Joe Reis) - M4: serving & viewsPart 3 · serving surfaces and the semantic contract
365DS Intro to Data Warehousing - S6: warehouse to business valueWhole session · the warehouse finally meets its consumers
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · A view differs from a materialized view how?

The triangle: freshness, cost, speed - pick two. Views maximize freshness, materialized views maximize read speed, and the refresh schedule is where the staleness hides.

2 · Why do marts encode business policy (like "finance excludes refunds") in their definition?

Three teams re-implementing one revenue rule produces three Monday numbers. Baking the rule into the mart makes the policy a fact of the data, not a habit of the people.

3 · GROUP BY ROLLUP (month, channel) returns...

ROLLUP adds the subtotal altitudes to the ordinary GROUP BY output, in one scan. NULL in a rolled-up column reads as "all" - the SQL residue of the old OLAP cube.

Builder session 7 cheat sheet · pin this

Data martA team-shaped slice of the core star with business policy written once, inside the definition. Consumers read marts, never raw staging.
Policy in one placeThe mart's WHERE clause is the business rule. Finance excludes refunds by definition, not by analyst discipline.
ViewSaved query, computed per read. Always fresh, zero storage, pays compute every open. The right default.
Materialized view / tableStored results, refreshed on schedule or by your b6 load. Instant reads, staleness debt. Promote when hit rates demand it.
ROLLUPGROUP BY ROLLUP (a, b) = detail + subtotals + grand total in one scan. NULL in a rolled column means "all".
Pre-aggregate whenThe same aggregate is read hundreds of times a day. One-off exploration hits the star directly instead.
Semantic contractA mart is a promise: columns stay, grain is stated, definitions documented in business words. Breaking it breaks dashboards silently.
Marts minimizeEvery column is a disclosure decision. Names and fine-grained locations stay out of wide-access marts.