Contracts, docs and deployment

A contract is two things in the model's YAML: contract: enforced: true in the config, and a data_type on every column. Once both are present, dbt requires the model's compiled SELECT to produce exactly those columns with exactly those types. For mart_merchant_daily that promise is activity_date as DATE, merchant_id as VARCHAR, net_revenue as DOUBLE, aov_numerator as DOUBLE and aov_denominator as BIGINT.

Now suppose somebody tidies the model and renames net_revenue to revenue, and a round() turns aov_denominator into a DOUBLE on the way past. Three things happen, in this order:

• The build fails at that model, before any object is created. There is no half-correct table for a dashboard to read.
• The error names the columns. dbt reports the mismatch per column - what was promised, what the SELECT produced - so the diagnosis takes seconds rather than an afternoon.
• Everything downstream SKIPs, exactly as it does for a failed test. The blast radius is contained by the same graph.

Constraints go one step further and push the promise into the warehouse itself: not_null, primary_key, unique and check can be declared alongside the types. How much of that is actually enforced depends on the warehouse - some platforms enforce all of it, some only not_null, and DuckDB sits in the middle. dbt is explicit about this in its docs and you should be explicit about it with your team: a constraint your platform does not enforce is documentation, and calling it enforcement is how people stop trusting the word.

A contract is a promise, and a promise you make everywhere is a promise you will resent. The useful rule is that a contract belongs on a model with a consumer you cannot edit:

• Yes on marts that a BI tool, an export or a reverse-ETL sync reads. These are the models where a rename is somebody else's outage. In this project that is the three mart_* models.
• Yes on anything another team refs across a project boundary. If they cannot see your pull request, they need an interface, not goodwill.
• No on staging models. They are cheap, disposable and consumed only by models in the same repo, where the DAG already tells you what breaks and CI already tests it.
• No on intermediate models. They are ephemeral - there is no object, so there is nothing to have a shape.

The cost is real and it is not compute: it is that every column addition now touches two files, and a type change becomes a conversation. That is the correct amount of friction on a public interface and the wrong amount everywhere else. Model versions exist for the case where you need to change a contract without breaking today's consumers - you publish the new shape alongside the old one and migrate readers deliberately.

A contract guarantees that net_revenue exists and is a DOUBLE. It guarantees nothing about what net_revenue means. Somebody can change the model from settled revenue to gross revenue, keep the name and the type, and the contract will be perfectly satisfied while every dashboard on the other side becomes wrong by 8.1%.

That failure is the b7 lesson again, and it needs the b7 tool: a singular test comparing the mart's total against the fact it aggregates. Contracts and tests are complements, not alternatives - shape and meaning are two different promises, and a mature model layer makes both of them explicitly.

Session b4 ended on a blind spot: the graph knows every dependency you expressed with ref() and nothing about a dashboard, an export or a reverse-ETL sync. An exposure is a few lines of YAML that puts those back in the graph. Each one names the consumer, its type (dashboard, notebook, analysis, ml, application), an owner with an email, and its depends_on list of ref() calls.

Bazaar has four such consumers, and writing them down changes three things:

• They appear in the lineage and in the generated docs, so a person reading mart_merchant_daily can see that a finance export and a CRM sync read it, with a name to contact.
• They become selectable. dbt build --select +exposure:exec_funnel builds and tests everything that dashboard depends on and nothing else - which is the right command before a board meeting, and the right command when somebody asks "is the exec dashboard safe to look at".
• An owner exists. The most useful field in an exposure is not technical. It is the person you have to tell.

dbt docs generate produces two JSON files in target/ and a static site that renders them:

• manifest.json - every node, its description, its columns, its tests, its config, and the full parent and child maps. This is the project as data: 32 nodes and 47 edges for Bazaar, derived from the ref() calls and nothing else.
• catalog.json - what the warehouse actually has: real column names, real data types, row counts, sizes. Queried from the warehouse after the build, not inferred from your SQL.

The site is the least interesting output. The point is that both files are machine-readable, so anything can read them: a catalog tool, a lineage tool, a Slack bot, a CI check that fails when a mart has no description. This course is itself an example. The lineage explorer in session b4 is not a picture anyone drew - it reads this project's manifest.json, and the raw-versus-compiled SQL on every node comes from the same place. That is the difference between documentation that drifts and documentation that is generated: the second kind cannot disagree with what was built, because it is what was built.

The one thing generation cannot do is write the description. A model with no description generates beautiful, empty documentation. Descriptions are the human half, and they are the half that decays.

A target is a named set of connection details in profiles.yml: which database, which schema, how many threads. This project has dev; a real deployment adds prod with a service account and the production schema. Because every model says ref() instead of a table name, the same code builds into your personal schema or the production one depending on a single flag. There is no branch, no environment variable inside the SQL, and no diff between what you tested and what ships.

That leaves the question of how CI stays fast. Rebuilding 23 models takes 1.22 seconds here and forty minutes on a real project, so testing everything on every pull request is how CI becomes something people learn to skip. state:modified+ is the answer:

• CI downloads the manifest.json from the last successful production build - the "state".
• --state path/to/prod/manifest --select state:modified+ compares the branch to it and selects the nodes whose definition changed, plus everything downstream of them.
• dbt builds and tests exactly that subgraph. A one-line change to a mart tests that mart; a change to stg_bazaar__orders tests eight nodes, because the graph knows.

Two things make this bite in practice. First, you need somewhere to keep the production manifest - an artifact store, a bucket, whatever your CI already has. Second, a change to dbt_project.yml or a macro can modify far more than it looks like, so the selector is honest about being large sometimes. That is the correct behaviour and worth explaining to whoever asks why one CI run took twenty minutes.

These are Bazaar's four consumers, written as an exposure list with the columns each one reads. The join against information_schema is a check nobody usually writes and everybody needs: does every column an exposure claims actually exist in the model it depends on?

WITH exposures(exposure_name, exposure_type, depends_on, column_used) AS (
    VALUES ('merchant_scorecard', 'dashboard',   'mart_merchant_daily',      'net_revenue'),
           ('merchant_scorecard', 'dashboard',   'mart_merchant_daily',      'merchant_tier'),
           ('exec_funnel',        'dashboard',   'mart_funnel_daily',        'carts_converted'),
           ('exec_funnel',        'dashboard',   'mart_funnel_daily',        'net_revenue'),
           ('ops_deploy_window',  'dashboard',   'mart_hourly_transactions', 'attempts'),
           ('crm_sync',           'reverse_etl', 'mart_merchant_daily',      'merchant_id'),
           ('finance_export',     'export',      'mart_merchant_daily',      'commission_amount')
)
SELECT e.exposure_name, e.exposure_type, e.depends_on, e.column_used,
       CASE WHEN c.column_name IS NULL
            THEN 'BREAKS - column not in the model' ELSE 'ok' END AS status
FROM exposures e
LEFT JOIN information_schema.columns c
       ON c.table_schema = 'main_marts'
      AND c.table_name   = e.depends_on
      AND c.column_name  = e.column_used
ORDER BY status, e.exposure_name;

In a real project this check belongs in CI, and it is the difference between an exposure that documents and an exposure that protects. Note also what the result gives you for free: dbt build --select +exposure:finance_export is now the command that rebuilds and tests exactly what that export depends on.

dbt docs generate builds catalog.json by asking the warehouse what it actually has. Here is the same question, asked directly. Twelve mart objects, every column, every type - and the fact that this is a query rather than a document is the whole idea behind docs-as-data.

SELECT table_name,
       count(*) AS columns,
       sum(CASE WHEN data_type LIKE '%INT%' OR data_type = 'DOUBLE'
                THEN 1 ELSE 0 END) AS numeric_columns,
       sum(CASE WHEN data_type = 'VARCHAR' THEN 1 ELSE 0 END) AS text_columns,
       sum(CASE WHEN data_type IN ('DATE', 'TIMESTAMP') THEN 1 ELSE 0 END) AS date_columns
FROM information_schema.columns
WHERE table_schema = 'main_marts'
GROUP BY table_name
ORDER BY table_name;

A CI check built on this shape is three lines long and catches real problems: a mart with a column count that changed unexpectedly, a fact table that acquired a text column, a dimension that lost its date. None of that needs a data-catalog product.

The selector walks edges. Here are real edges from this project's manifest.json, and a recursive CTE doing exactly the walk dbt does when you change one staging model. The answer - eight nodes including the modified one - is what CI would build and test on that pull request, and nothing else.

WITH RECURSIVE edges(parent, child) AS (
    VALUES ('stg_bazaar__orders', 'fct_carts'),
           ('stg_bazaar__orders', 'fct_transactions'),
           ('stg_bazaar__orders', 'int_order_lines_enriched'),
           ('stg_bazaar__carts',  'fct_carts'),
           ('int_cart_contents',  'fct_carts'),
           ('int_order_lines_enriched', 'fct_order_items'),
           ('fct_carts',        'mart_funnel_daily'),
           ('fct_order_items',  'mart_funnel_daily'),
           ('fct_order_items',  'mart_merchant_daily'),
           ('fct_transactions', 'mart_funnel_daily'),
           ('fct_transactions', 'mart_hourly_transactions')
),
downstream(node, hops) AS (
    SELECT 'stg_bazaar__orders', 0
    UNION
    SELECT e.child, d.hops + 1
    FROM edges e JOIN downstream d ON e.parent = d.node
)
SELECT node, min(hops) AS hops_away
FROM downstream
GROUP BY node
ORDER BY hops_away, node;

Change the starting node to fct_order_items and the selection drops to three. That difference is why state:modified+ keeps CI in minutes on a project where a full build takes an hour, and why the graph you got for free in b4 turns out to be the thing that makes deployment affordable.

Twenty objects in two schemas, and the schema names are the only trace a target leaves. Point the same code at prod and this result is identical except for the prefix - which is the strongest argument for ref() there is.

SELECT table_schema, table_type, count(*) AS objects
FROM information_schema.tables
WHERE table_schema IN ('main_staging', 'main_marts')
GROUP BY 1, 2
ORDER BY 1, 2;

Twenty objects for twenty-three models, because the three intermediate models are ephemeral. Worth remembering when somebody asks why the docs site lists more models than the warehouse has tables.