The org question, and the honest close
Data titles blur together in job ads, and that costs you real money - you over-hire senior data scientists to do plumbing work, or ask one data engineer to also be a modeler, an ops team, and a stats expert. This final session untangles the roles, shows how data engineering feeds every downstream consumer, and gives you a staged roadmap so you build the capability in the right order. It closes with an honest list of what this course did not cover and exactly where to go next.
Who does what on a data team 8 min live
There are five roles you will keep hearing, and they sit in a line from raw data to business value. Data engineers build the pipelines. Analytics engineers model the landed data closer to the warehouse. ML engineers put models into production. Data scientists find the insight and build the models. Platform engineers keep the infrastructure standing. Data engineering feeds every one of the others - it is the layer they all stand on.
LiveThe five roles, and where they differ3 min▶
The clearest way to tell them apart is to ask what they hand off, and to whom.
- Data engineer: builds and runs the pipelines that get data from sources into usable, served form. Closest to the raw plumbing. Feeds everyone below.
- Analytics engineer: takes landed data and models it (often in dbt) into clean, tested, business-ready tables. Sits between DE and the warehouse - the bridge role.
- ML engineer: puts models into production reliably - the DE mindset applied to model serving.
- Data scientist: finds insight, runs experiments, builds the models. Consumes what DE and analytics engineering prepare.
- Platform / infra engineer: keeps the underlying systems running so all of the above have something to build on.
Self-studyThe analytics engineer, and why it matters to you2 min read▶
The analytics engineer is the newest role and the one leaders most often miss. They live in the seam between data engineering and the warehouse: they take the raw tables DE lands and turn them into modeled, documented, trustworthy datasets using SQL and dbt - the modeling work owned by the learn-data-warehouse course. Why it matters: hiring one analytics engineer often unblocks a whole team of analysts and data scientists who were otherwise wrangling raw data themselves. It is frequently the highest-leverage data hire a growing company can make, and it is the role that most directly connects the plumbing you learned about to the business questions your leaders actually ask.
The senior DS doing junior plumbing. A company hired three expensive data scientists and watched them spend most of their time cleaning and joining raw exports instead of modeling. One analytics engineer, added to build the clean shared tables, freed all three to do the work they were hired for. The bottleneck was never talent - it was a missing role in the middle of the line.
The crawl-walk-run roadmap 7 min live
You do not build a data platform all at once, and you should not try. The teams that succeed earn each stage before the next. Crawl: get one reliable daily batch feed and one clean dataset people trust. Walk: make loads incremental, tune formats and tiers, agree contracts with sources. Run: add streaming where it pays for itself, and let teams self-serve. Skipping ahead - streaming before you have reliable batch - is the most common and expensive mistake.
LiveWhat each stage looks like3 min▶
The stages map directly onto everything the earlier sessions covered.
- Crawl: one source ingested on a reliable daily schedule, landed, transformed, and served as a single dataset the business actually trusts. Unglamorous and worth more than any dashboard demo.
- Walk: loads become incremental (only new data, not full reloads), formats and tiers get tuned for cost (session a5), and you agree contracts with source teams so schema changes stop breaking you.
- Run: streaming is added only where real-time genuinely pays (session a3), and pipelines become self-serve so analysts and DS build their own without waiting on the DE team.
Self-studyThe most expensive mistake: skipping stages2 min read▶
The pattern is predictable. A leader sees a competitor's real-time dashboard, or reads about streaming, and asks the team to build it - while the nightly batch job still fails twice a week and no one trusts the numbers. Streaming on top of an unreliable foundation is not faster insight; it is unreliable insight arriving faster. The discipline is to ask "which stage are we honestly in" and fix the current stage's weakness before buying the next stage's tools. Crawl reliably, then walk, then run. Almost every "our data platform is a mess" story is really a team that tried to run before it could crawl.
Streaming before reliable batch. A retailer invested a quarter in a real-time pipeline because the board wanted "live" numbers. Meanwhile the daily batch that fed the actual financial reports kept failing silently. They had a fast, shiny stream and a finance team that still did not trust the totals. Fixing the boring daily batch first would have delivered more value for a fraction of the spend.
The honest map of what is next 6 min live
Data engineering does not end the story - it hands the baton. Once data is served, the warehouse models it, DataOps keeps the pipeline running, and ML and DS consume it. This course deliberately stayed in its lane, and honesty means naming what it did not cover and pointing you at the sibling courses that do.
| Once DE has served the data | Who takes it | Go to |
|---|---|---|
| modeling into facts, dimensions, marts (star schema, SCD) | analytics engineer | learn-data-warehouse |
| orchestration, CI/CD, monitoring, data-quality testing | DataOps / platform | learn-dataops |
| writing the queries and analysis on top | analysts, data scientists | learn-sql |
LiveWhat this course did NOT cover3 min▶
Honesty about scope is a leadership tool - it stops you expecting one course, or one hire, to do everything. Deliberately left out of data engineering, and owned elsewhere:
- Orchestration, CI/CD, monitoring: scheduling pipelines, automated testing, alerting, infrastructure as code - all the "keep it running reliably" work lives in learn-dataops.
- Dimensional modeling: star schemas, slowly changing dimensions, marts, warehouse tuning - the shape data takes inside the warehouse - is learn-data-warehouse.
- SQL as a language: the query syntax itself is learn-sql. This course used SQL but did not teach it.
Self-studyYour quarterly questions and the road ahead2 min read▶
Four questions to bring to your team every quarter: Which crawl-walk-run stage are we honestly in, and what is the one thing blocking the next? Are we missing the analytics-engineer seam between our pipelines and our warehouse? Is our storage cost and format still under control (session a5)? And are we building any real-time capability before our batch foundation is trustworthy (session a3)? For where to go next as a leader, the sibling courses in the table above cover the handoffs; the builder track (b1-b10) of this same course goes hands-on for anyone on your team who wants to actually construct the Daybreak pipeline. You have finished the leader track - you now have the vocabulary to lead a data engineering effort without having to write a line of it yourself.
Take this back to your team ◐ 20-30 min total
- Map your current data team onto the five roles. Which role is missing or doubled up? Is anyone doing plumbing who should be doing insight?
- Decide, honestly, which crawl-walk-run stage you are in. Name the single thing blocking the next stage.
- Check whether you have an analytics-engineer seam between pipelines and warehouse. If not, ask whether one hire would unblock several people.
- Review the "what this course did not cover" list and match each gap to a sibling course or a role you need.
- Book the four quarterly questions into a recurring data-team review, and point any builders on your team at the b1-b10 track.
Official sources covered
This session teaches the org-and-roadmap core of the DeepLearning.AI Data Engineering certificate (Joe Reis) and Reis & Housley's Fundamentals of Data Engineering, framed for leaders staffing and sequencing the work. Verified as of mid-2026 - re-verify role definitions and tooling as the field evolves. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · What does an analytics engineer do that a data engineer typically does not?
The analytics engineer sits in the seam between pipelines and warehouse, turning raw landed tables into modeled, trustworthy datasets. Hiring one often unblocks a whole team of analysts and data scientists.
2 · In the crawl-walk-run roadmap, what belongs to "crawl"?
Crawl means earning a reliable foundation first: one dependable daily batch, landed and served as a trusted dataset. Streaming and self-serve are the "run" stage, earned only after walk.
3 · A team asks to build real-time streaming while the nightly batch job still fails twice a week. What is the leadership move?
Skipping stages is the classic expensive mistake. Earn reliable batch (crawl) before adding streaming (run). Almost every "our platform is a mess" story is a team that ran before it could crawl.