The last mile
RetailPulse can do everything except the thing that scares people most: go to production on purpose, safely, on a random Tuesday. Seven sessions built the pieces. Tonight assembles them into a release. The hard truth of DataOps - the reason this is the last and hardest session - is that a data product is not one artifact. It is code, a schema, and a model, and they must all promote together, in the right order, or the release corrupts data instead of shipping value. This is the diff at full size: one change, touching everything, that either passes every gate or does not ship. And there is a final twist - the last change you review tonight will have been written by an AI agent, not you, because that is the job the rest of this decade actually is.
Environments: dev -> staging -> prod 6 min live
Production is not where you experiment. A change earns its way there through environments that get progressively closer to real: dev, where you build and break things; staging, a faithful mirror of prod where you prove the whole release end to end; and prod, where real users and real money live. What promotes between them is not just code - it is the code, the data config, the model version, and the schema, moving together.
LiveEnvironment parity: why staging must mirror prod3 min▶
Staging exists for one reason: to let you find out what a release does before real users do. That only works if staging is a faithful copy of prod. The moment staging drifts - a different Postgres version, a smaller dataset, a missing env var - it starts lying to you, and a green staging run stops meaning "prod will be fine."
- Same shape, smaller scale. Staging runs the same images, the same schema, the same migrations, and a representative sample of real data - never a toy dataset that hides volume and distribution bugs.
- Same config mechanism. Staging and prod read secrets and settings the same way (env vars from a secret manager), so "it worked in staging" is a real signal, not luck.
- Promotion, not rebuild. You promote the same built image from staging to prod. Rebuilding for prod reintroduces the "works in staging" gap you just closed.
- Dev is disposable, prod is precious. dev optimizes for speed and iteration; prod optimizes for safety and reversibility. Staging is the bridge that keeps the two honest.
The migration that only broke at scale. A team tested a schema change on a 500-row staging table. Green. In prod the same migration locked a 40-million-row table for eleven minutes and took checkout down. The bug was never in the code - it was in a staging environment that did not mirror prod's data volume. Parity is not pedantry; it is the difference between a caught bug and an outage.
Self-studyHow many environments do you actually need2 min read▶
Three (dev, staging, prod) is the sensible default for a data team. Smaller shops sometimes collapse dev and staging; larger ones add a pre-prod or a per-developer ephemeral environment spun up from the same IaC. The number matters less than the principle: at least one environment between "my laptop" and "real users" that is close enough to prod to trust. If you only have laptop and prod, prod is your staging - and your users are your testers.
Releasing data, model, and database together 6 min live
Here is what makes DataOps genuinely harder than DevOps. Shipping an app is shipping code. Shipping a data product is shipping a pipeline change, a schema migration, and a model version - three artifacts that depend on each other and must promote as one unit, in the right order. Get the order wrong and you deploy code that reads a column the migration has not added yet, or serve a model trained on a schema prod does not have.
LiveThe lockstep problem, and the order that solves it3 min▶
The hard part of DataOps is that the three artifacts are coupled. The pipeline writes a new column; the migration must create it first; the model expects it to exist. Deploy them in the wrong order and something reads a column that is not there yet. The safe sequence is expand -> deploy -> backfill -> switch, built on the expand/contract migration pattern from b4:
- Migrate (expand): add the new column or table - additive only, nothing dropped. Old code keeps working because you have taken nothing away.
- Deploy: ship the new pipeline and model, which can now use the new column. Both old and new schema coexist for this window.
- Backfill: populate the new column for historical rows so the model sees complete data.
- Switch: cut traffic to the new version. Only now, in a later release, do you contract - drop the old column once nothing reads it.
Wrap the switch in a safety pattern so a bad release is reversible: blue-green (run new beside old, flip the pointer, flip back instantly on trouble), canary (send 5% of traffic first, watch, then ramp), or shadow (run the new model on real traffic without serving it, compare, then promote). All three share one goal: never make production the first place a release runs unguarded.
Self-studyBlue-green vs canary vs shadow - which when2 min read▶
Blue-green is simplest and best when you can hold two full copies and want instant rollback - flip a pointer, done. Canary fits when you want to limit blast radius and have good metrics to watch as you ramp. Shadow is the model-specific gem: run the new forecast model on real traffic without serving its output, compare predictions to the live model and to what actually happened, and promote only when it wins. For RetailPulse's demand forecast, shadow is how you prove a new model is better on real data before a single customer sees it.
The graduation checklist 4 min live
Step back and look at what RetailPulse is now. Every capability that separates a hobby script from an industrialized data product is present and connected. This is the DataOps scorecard - the one you can hold up in any review to say "this data product is mature." Session by session, you built each row by hand.
LiveThe DataOps scorecard RetailPulse now passes3 min▶
| Capability | What it means | Built in |
|---|---|---|
| Version control | All code, config, SQL in Git; protected main | b1 |
| CI | Lint + test gate on every PR | b1 |
| Orchestration | Pipeline scheduled and retried as a DAG | b2 |
| Data tests + contract | Great Expectations suite + a data contract | b3 |
| Versioned schema | Migrations, reversible, in Git | b4 |
| Tracked + registered model | Experiments logged, model in a registry | b5 |
| Serving + monitoring | FastAPI /predict, Evidently drift, retrain trigger | b6 |
| Observability + alerts | Structured logs, freshness/volume, routed alerts | b7 |
| Secrets | Out of Git, in .env / a secret manager | b7 |
| Promotion to prod | Gated CD, staging parity, safe lockstep release | b8 - tonight |
Ten rows. Every one green. That is not a demo - that is a production-grade data product, and you can defend every line of it because you built it yourself and felt it break first.
Self-studyWhat a mature next step looks like2 min read▶
RetailPulse v1.0 is a real, defensible platform - but there is always a next altitude. When you are ready:
- Feature store (Feast): so the features the model trains on and serves on are provably the same, killing training-serving skew for good.
- Streaming: move from a nightly batch to near-real-time ingestion (Kafka, Kinesis) when the business needs fresher-than-daily.
- Full IaC: promote your Docker Compose to Terraform-managed cloud infra, so prod itself is reproducible from code, not clicks.
None of these are prerequisites for shipping. They are the direction of travel once v1.0 is live and earning trust. Ship first, then climb.
A promotion workflow ★ 12 min · everyone builds
RetailPulse v1.0, part one: a GitHub Actions CD workflow. On every merge to main it builds, tests, and pushes the image, then deploys to staging automatically. On a version tag it promotes that same image to prod - but only after a human clicks approve on a protected environment. Automation for speed, a gate for safety.
Create a staging and a production environment in GitHub (Settings -> Environments). Add a required reviewer to production - that reviewer is the manual approval gate.
Write .github/workflows/cd.yml (Prompt A). The deploy-staging job triggers on push to main: build, test, push the image, deploy to staging.
Add a promote-prod job that triggers only on a tag matching v* and targets the production environment - so it pauses for the required-reviewer approval before it runs.
Promote the same image by digest, not a rebuild. Rebuilding for prod would reintroduce the staging-parity gap from Part 1.
Merge a trivial PR and watch it flow to staging with no human. Then push a tag and watch prod wait on the approval button. That pause is the whole point.
End-to-end release and tag v1.0 ★ 10 min · everyone builds
RetailPulse v1.0, part two: run the entire product end to end in staging - migrate, ingest, test, train and register, deploy, monitor - verify it, then tag v1.0 and let the gated pipeline promote it to prod. This is the moment every session has been building toward.
In staging, run the release order from Part 2: alembic upgrade head (expand), trigger the Airflow DAG to ingest, let Great Expectations gate quality, retrain and register the model in MLflow.
Deploy the new image to staging, hit /predict, and confirm the forecast is sane. Check the b7 freshness and volume alerts stay quiet - a green observability layer is part of "verified."
Run the end-to-end smoke script (Prompt B) that walks the whole chain and fails loudly on any broken link. Green means the release is real, not hopeful.
Tag it: git tag v1.0 && git push origin v1.0. The CD workflow's promote-prod job fires and pauses on the approval gate.
Approve the promotion. RetailPulse v1.0 is in production - code, schema, and model, promoted in lockstep, reversible if anything turns red. You shipped. You graduated.
The release that stopped being scary. A retail data team used to schedule "the model update" as a quarterly all-hands event with a rollback runbook and crossed fingers. After they built exactly this - staging parity, a lockstep release order, a gated CD pipeline with blue-green switch - a model promotion became a Tuesday afternoon tag and an approval click. The technology did not just make releases safer; it made them boring. Boring releases are the real trophy of DataOps.
Review a change you did not write ★ 10 min · the real capstone
Here is the quiet truth of everything you built. By the time this course is popular, an AI agent will scaffold an Airflow DAG, write the dbt tests, and generate the migration faster than you can open the file. So the skill that pays your salary is not typing the pipeline - it is being the one who can read the change the machine produced and tell whether it is safe to merge. Every gate you built across b1-b7 exists for this moment. The generator is cheap and confidently wrong; the verifier is scarce. Tonight you are the verifier.
Ask an AI coding agent to add a feature to RetailPulse - for example, "add a promo_flag column and use it in the demand forecast". Let it write the migration, the pipeline change, the test, and the model tweak. Do not review it yet.
Open the change as a pull request and let your gates run: CI, the Great Expectations suite, the data-contract check, the migration dry-run, the model-CI metric. Watch which gates the agent's change passes and which it trips.
Now read the diff like a skeptic. Look for the failure modes agents are good at hiding: a silent type coercion, a join that fans out row counts, a migration that is not additive (breaks expand-contract), target leakage in the new feature, a dropped null-check. Find at least one thing the green checks did not catch.
Write the review: what you would request-change, what you would block, what you would let merge. This is the artifact - not the code the agent wrote, but your judgment about it.
Only if it survives your review and every gate: merge it, and run it through the same gated CD you built in Demo 1. The agent proposed; the gates filtered; you decided. That sequence is the job.
The agent that shipped a data leak nobody caught for a sprint. A team let an assistant add a "days since last purchase" feature to a churn model. Every test was green; the model's accuracy jumped, so it merged. The feature was computed after the churn event - textbook leakage - and the model was quietly useless in production for two weeks. No linter catches that. A human who understood the pipeline would have, in thirty seconds. That human is what this course actually trains you to be.
Try it yourself - graduate RetailPulse ◐ 45-60 min total
- Finish the capstone: run
release_e2e.shgreen in staging, then tagv1.0and approve the promotion. Cross every box on the capstone checklist yourself. - Break the gate on purpose: push a tag with a failing smoke test and confirm the promotion never reaches prod. A gate you have not tested is not a gate.
- Add a one-line rollback to
deploy.sh(re-point to the previous image) and rehearse it once. The confidence to ship comes from the confidence to un-ship. - What to build next: pick one from the mature-next-step card - a Feast feature store, a streaming source, or Terraform for prod - and sketch how it slots into RetailPulse. You now have the spine to hang it on.
- Write your own one-paragraph "DataOps scorecard" for a real project at work. You now know all ten rows and can tell which are green.
- The verifier drill: have an AI agent generate one more RetailPulse change, then write a real code review of it - what you would block and why. Practice being the human the gates cannot replace. This is the skill that outlives every tool in this course.
What this session covers
This track teaches DataOps on an open-source stack from the tools' official docs, with AWS mapping notes. This capstone covers:
Three questions before you graduate 🎓 ◐ 90 seconds
1 · Why must staging mirror prod as closely as possible?
A migration tested on 500 staging rows can lock a 40-million-row prod table. Parity - same images, schema, config mechanism, representative data - is what makes "it worked in staging" a real signal instead of luck.
2 · What makes releasing a data product harder than releasing an app?
Ship them out of order and code reads a column the migration has not added, or a model serves on a schema prod does not have. The lockstep and the ordering are the hard, DataOps-specific part.
3 · What does the manual approval gate on the production environment give you?
Merge to main deploys to staging automatically; a version tag pauses on a required reviewer before prod. Automation handles the repetitive work; the gate keeps a human in the loop for the one step that touches real users.