Where RetailPulse is tonight
Last session you gave RetailPulse a demand-forecast model and did it properly: every training run tracked in MLflow, the winner registered as demand_forecast and promoted to Staging, and a model-CI gate that blocks a regressing model from merging. What you do not have yet is a way for anything to actually use that model, or to notice when it quietly goes stale. Tonight closes both gaps and closes the loop back to b5's training code.
/predict endpoint that loads the registered Production model and returns a demand forecast, containerized in a Dockerfile. An Evidently data-drift report that compares recent sales to the training reference. And a drift-triggered retrain rule - an Airflow task that, when drift crosses a threshold, kicks off b5's training run again. That is RetailPulse v0.6, and it is a full MLOps loop.
Batch vs online serving 6 min live
There are two ways to get predictions out of a model, and picking wrong makes the whole system awkward. Batch scores a big set of rows on a schedule and writes them somewhere. Online answers one request at a time over an API, in milliseconds. Most demand-forecast use cases are naturally batch - but the moment someone wants a prediction on demand, you need an endpoint too.
LiveWhen each mode fits - and why RetailPulse picks online3 min▶
The two modes are not rivals; they answer different questions:
- Batch scoring: a scheduled job scores a large set at once and writes results to a table or file. Cheap, simple, no live infrastructure. Perfect when consumers read predictions on their own cadence - a nightly demand forecast feeding tomorrow's restock plan.
- Online serving: a REST endpoint returns a prediction per request in milliseconds. Needed when the answer is required interactively - a planner tweaks an assumption in a dashboard and wants the forecast to update now.
- Latency vs throughput: batch optimizes throughput (millions of rows, minutes are fine); online optimizes latency (one row, milliseconds matter).
- RetailPulse's choice: we build the online FastAPI endpoint tonight because it is the harder, more reusable pattern - and we note the batch option, which is just calling the same model inside a scheduled Airflow task like the one from b2.
The endpoint that should have been a batch job. A team wrapped a demand model in a low-latency service that got called once a night by a single scheduled report. They paid for an always-on endpoint, health checks, and autoscaling to serve one request a day. The forecast never needed to be online - a batch job writing to a table would have been a tenth of the cost and half the pages. Match the serving mode to how the prediction is actually consumed.
Self-studyWhere features come from at serving time2 min read▶
A subtle trap: the model was trained on features like lag_1 and roll_3, computed from history. At serving time those features must be computed the exact same way, or you get training/serving skew - the model sees inputs it never learned on. For batch this is easy: the same feature code runs over the same table. For online it is harder: the request must either carry the features or the endpoint must look them up. Tonight the request carries them, which keeps the demo honest; a feature store (like feast) is the grown-up answer when lookups get complex.
Serve the model with FastAPI + Docker 6 min live
An online endpoint is a small, well-behaved web service: a request comes in, you validate it, load the registered Production model, predict, and return the number. FastAPI gives you typed request validation for free via Pydantic, and Docker - the same skill from b1 - makes the service run identically on your laptop, in CI, and in prod.
LiveLoad from the registry, validate the input, return the number3 min▶
Three design choices make this endpoint production-grade rather than a toy:
- Load by registry reference, not a path: ask MLflow for
models:/demand_forecast/Production. When you promote a new version in the registry (the b5 skill), the endpoint picks it up on restart - no code change, no redeploy of new logic. - Validate with Pydantic: the request body is a typed model. A missing field or a string where a number belongs gets a clean 422 error, not a stack trace deep in the predict call. Bad input never reaches the model.
- Load once, not per request: load the model at startup and keep it in memory. Loading on every request would add hundreds of milliseconds and hammer the registry.
- Containerize: wrap it in the same
python:3.11-slimDocker pattern from b1. The image is the unit you deploy to ECS, a VM, or Kubernetes.
Monitor for drift and retrain 6 min live
A deployed model degrades not because the code breaks but because the world moves. Last quarter's buying patterns are not this quarter's. Monitoring watches the live data and predictions, detects when they have drifted away from what the model was trained on, and triggers a retrain before the forecast quietly rots. This is the piece that makes the loop a loop.
LiveData drift vs concept drift, and monitoring with Evidently3 min▶
Two different things go wrong after deployment, and it helps to name them:
- Data drift: the input distribution changes. A new product line, a promotion, a seasonal shift - the features arriving now look statistically different from the training features. The model may still be right in principle, but it is extrapolating.
- Concept drift: the relationship between inputs and target changes. The same features now imply different demand - a change in customer behaviour, a competitor, a price war. Even perfect input monitoring will not catch this directly; you catch it by watching prediction quality against actuals.
- Monitoring with Evidently: the open-source default. Point it at a reference dataset (your training data) and a current dataset (recent production data) and it produces a data-drift report - per-feature drift and an overall drift share - plus prediction monitoring. It is the measuring instrument; your threshold is the decision.
| What drifted | How you notice | Response |
|---|---|---|
| Input features (data drift) | Evidently drift share crosses threshold | Retrain on recent data |
| Input→target link (concept drift) | Live RMSE vs actuals degrades | Retrain, maybe re-feature |
| Nothing - just noise | Drift share below threshold | Do nothing; do not chase noise |
Self-studyRetrain triggers: scheduled vs metric vs drift2 min read▶
There are three honest ways to decide when to retrain, and mature teams combine them:
- Scheduled: retrain every week or month regardless. Dead simple, predictable, but wasteful when nothing changed and too slow when something changes fast.
- Metric-based: retrain when live prediction quality (RMSE against actuals, once they arrive) drops past a threshold. Directly tied to what you care about, but needs ground-truth labels, which lag.
- Drift-based: retrain when input drift crosses a threshold - the earliest signal, available before labels do. What you build tonight. The risk is chasing drift that does not actually hurt accuracy, so pair it with the metric check when labels arrive.
Whichever trigger fires, it should call the same training code from b5 - tracked, evaluated, gated by model CI, and registered. Retraining is not a special path; it is the b5 pipeline run again on newer data.
A FastAPI prediction service ★ 12 min · everyone builds
Build serve.py: a FastAPI app that loads the registered Production demand-forecast model once at startup, exposes a typed /predict route, and returns the forecast. Then containerize it with the b1 Docker pattern and hit it with curl.
Add fastapi, uvicorn, and pydantic to pyproject.toml. Create src/retailpulse/serve.py.
At module load, pull the model from the registry: mlflow.pyfunc.load_model("models:/demand_forecast/Production"). Load once, reuse across requests.
Define a Pydantic ForecastRequest with the fields the model needs - product, region, lag_1, roll_3. FastAPI validates and rejects bad input automatically.
Write the POST /predict route: turn the request into a one-row DataFrame, call model.predict, return the number as JSON. Add a GET /health that returns ok.
Run it with uvicorn, curl the endpoint, then write the Dockerfile and docker build && docker run -p 8000:8000. Same prediction, now in a container.
models:/demand_forecast/Production means promoting a new version in the MLflow registry - the b5 workflow - is the entire deploy. The endpoint just needs a restart, and rollback is re-promoting the previous version. Same code, no scramble.
Drift monitoring + retrain trigger ★ 10 min · build your own
Generate an Evidently data-drift report comparing recent sales to the training reference, then write a rule that triggers a retrain when the drift share crosses a threshold - as an Airflow task that calls b5's training code. This is the last link that turns RetailPulse into a self-healing loop.
Add evidently to pyproject.toml. Create monitor.py with two frames: reference (the training slice) and current (recent production sales, same feature columns).
Run Evidently's DataDriftPreset over the two frames. Save the HTML report as an artifact and pull the overall drift share out of the report as a number.
Write the trigger rule: if drift_share > 0.5, return "retrain"; else "hold". This is deliberately simple and legible - a threshold you can defend.
Wrap it in an Airflow task (reusing the b2 DAG). A drift_check task runs the report on a schedule; a BranchPythonOperator routes to a retrain task when drift is high.
Make retrain call b5's train.main() on the newer data. The new run is tracked, gated by model CI, and registered - closing the loop back to session 5.
The model that was right until the promotion. A retailer's demand forecast performed well for months, then a big seasonal promotion changed buying patterns overnight. Nobody was watching the inputs, so the model kept confidently under-forecasting for three weeks - and stores kept running out. A drift report would have flagged the shift in days, triggered a retrain on the new pattern, and closed the gap. The model was never "broken"; the world moved and nothing noticed.
Try it yourself - this week ◐ 40-55 min total
- Finish both demos if you did not complete them live - especially the drift-to-retrain branch. Everyone should watch drift trigger a fresh, registered model at least once.
- Add a
/predict/batchroute to the FastAPI app that scores a list of rows, so RetailPulse has both serving modes from Part 1. - Force drift: shift the
currentdata (add a promotion spike) and confirm the drift share crosses the threshold and the DAG branches to retrain. - Wire the endpoint's model load to a
/healthcheck that also reports the loaded model version, so you always know what is live. - Optional reading: the FastAPI and Evidently docs, plus MLflow's model-serving guide - now a review of what you built by hand.
What this session covers
This track teaches DataOps on an open-source stack from the tools' official docs, with AWS mapping notes. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · When is batch scoring the right serving mode instead of an online endpoint?
Batch optimizes throughput for scheduled consumption; online optimizes latency for interactive requests. Match the mode to how the prediction is actually used, not to the model type.
2 · Why load the model as models:/demand_forecast/Production rather than a fixed file path?
Loading by registry stage decouples the serving code from any specific model version. Deploy and rollback happen in the registry, not in a redeploy of new logic.
3 · What is the difference between data drift and concept drift?
Data drift you catch by comparing input distributions (Evidently). Concept drift you catch by watching prediction quality against actuals - the same inputs now imply different demand.