Every system a number passes through

Take the month-end pack and the Monday report and go down them one number at a time. For each, ask the bank's question g05: "How is the report made today? Show me the file, and tell me who builds it." with its teach-back said first: "The current file is the spec. Every step done by hand is a step I have to reproduce exactly before I can improve it." She does not need to know what a source system is to answer. She knows that Priya downloads one CSV per outlet on Monday, 38 files, pastes them into a spreadsheet, and that a macro somebody wrote in 2019 merges them. Six hours.

The four answers carry four different builds. A number recorded by a system arrives on its own, in the same shape every time: sales at the till, one cloud POS vendor, a CSV per outlet, product codes that differ between Singapore and Malaysia. A number typed by a person arrives when somebody types it, in whatever shape they typed: the stock spreadsheet, in three copies. A number not recorded cannot be moved at all until somebody writes it down: supplier orders go over WhatsApp and there is no order table anywhere. And a number exported by a person on a schedule arrives as often as that person and their calendar allow: Priya's Monday files, and the bookkeeper in Kuala Lumpur, who exports once a month.

Say the fourth case out loud when you hear it, because it is the one a pipeline silently inherits. You can schedule a job to run every hour; it still reads a file that changed once a week or once a month. Ask g19 next: "How often does each system get exported today, and by whom?" with "How often a system is read today is a ceiling I inherit until a contract or a login changes it." At Harbourline the ceiling on finance's side is written into a contract: one export a month, and more is a contract change. That is a data-state fact, not a legal footnote, and it is why "pull everything daily" is a sentence you should never say before you have asked.

Two sources are usually run together into one number, and they should not be. In 2014 Steve Lohr reported in The New York Times that data scientists spend 50 percent to 80 percent of their time collecting and preparing data before it can be explored. In 2016 CrowdFlower's data science report found respondents spending 60 percent of their time cleaning and organising data and a further 19 percent collecting it. Two sources, two methods, two shapes of answer: one a range for collecting and preparing together, the other two separate figures. Quote whichever you use as it stands, and never fold them into one cleaning figure taken from the top of Lohr's range, because nobody measured that number.

The point for door 4 is the reason behind both. Preparation time is mostly the cost of the typed, the absent and the exported: reconciling three stock copies, finding out that refunds live only in the accounting system, discovering in week three that the finance export you planned to read nightly exists once a month. Every minute spent asking "where does this come from, and how does it reach you" is a minute not spent in week three finding out the hard way.

DAMA-DMBOK, 2nd edition, chapter 15, names five levels of data management maturity: Initial/Ad hoc, Repeatable, Defined, Managed, Optimized. Five, not six: DMBOK has no level 0. The names are the whole of what is borrowed, and they are reported from summaries rather than read from the book. The questions beside each rung are ours, written so a person who runs pharmacies can answer them about her own systems.

Ask them once per box on the map. The till: it records every sale the same way, so it is at least rung 2; whether anything about it is written down is a question you have not asked yet, so you do not place it higher. The ledger export: the bookkeeper sends it once a month by contract, the same way each time, rung 2 on what you know. The stock sheets: three copies in circulation and none declared real, rung 1. The Monday merge: 38 files pasted by hand and a macro somebody wrote in 2019, and the bottom-rung question gets the decisive answer, asked as g20: "When a number in the report is wrong, who fixes it, and does anyone find out?" Priya fixes it by hand in the spreadsheet; nobody is told; there is no record of what changed. Rung 1.

Write the rung beside each system in your notes and never tell her the number. "Your merge is a 1" is a grade, and a grade ends a conversation that was going well. What the reading gives you, as an engineer, is where the pipeline must not start: never copy the lowest rung faithfully and call it done. A job that reproduces the Monday merge on a schedule reproduces the silent fixes too, or loses them, and either way the number changes without a record. The real assessment, with the practices behind each level, is a governance discipline, taught in the Governance 101 course; in a discovery meeting the ladder is an interview aid and nothing more.

Joe Reis and Matt Housley, in Fundamentals of Data Engineering (O'Reilly, 2022), describe companies as starting with data, scaling with data or leading with data, reported here from summaries rather than a page. Harbourline is starting with data: one recorded source, one person who assembles the report, a bookkeeper on a monthly contract, and a request that is really a request to be able to answer an auditor. Whatever you propose should fit a company at that stage, which rules out most of what a platform vendor would propose.

Leandro DalleMule and Thomas H. Davenport, in "What's Your Data Strategy?" (Harvard Business Review, May-June 2017, reported), ask whether a company needs a single source of truth or several fit-for-purpose versions of the truth. Harbourline has both problems by accident: the POS and finance are two versions of revenue built along two paths, and the stock sheet is three versions of one thing with no agreed copy. The frame is useful because it names a choice. A company can decide to run two versions of revenue on purpose, one for Monday and one for the auditor; it should not discover it has.

The map starts with one question, g08: "Which systems does one sale touch, from the till to the bank to the ledger?" Say the teach-back first: "Every hand-off between systems is a place a number can change. I am counting the hand-offs." She can answer that without knowing a single technical word, and she answers in order: the till, the bank settlement, the ledger the bookkeeper keeps. That row alone explains most of the two percent: card settlements land two days late, cash is deposited weekly, and refunds exist only in the ledger. Two numbers built along two paths, disagreeing about timing before truth.

Then ask the same question of the Monday side and you get the second row: the till again, 38 CSVs pulled by Priya, her merge, the report. Notice what you do not get. Nobody has told you how a sale on the till ends up as a line in the stock spreadsheet: whether someone counts shelves, types from the till, or copies from last week. Draw that hand-off dashed and label it unknown. A dashed arrow on the map is not a gap in your drawing; it is the next question you ask.

Question g09 has to be asked with a name in the answer: "Who can log in to each of those, and pull an export?" Teach-back: "Access is the first blocker on most data projects, so I ask early and I ask by name." "IT can" is not an answer. "Kenneth can, and only Kenneth" is, and it is the answer that makes you ask what happens in January.

Kenneth's leaving date does not come out on the first question, whatever you ask. On the bench it sits behind a trust gate of 3, the highest on the sheet: she volunteers it only after enough open questions, each with its reason said aloud, have shown you are helping her and not auditing her, and every jargon question you ask on the way takes a point of that trust back. It is worth 4 points on the sheet, as much as the outcome itself. That is why the widest door is not the first one: one outcome question first, the month-end and Monday walk-throughs, the systems and the stock copies, and then the login, by name. Ask it cold and you get "our IT guy". Session 3 shows which meeting orders reach it in 45 minutes.

Question g11: "Where do the stock numbers live, and which copy is the real one?" Teach-back: "A file that exists three times has no real copy until someone declares one. I need that declared before I move it anywhere." At Harbourline the answer is three spreadsheets in circulation and one on Priya's laptop that she treats as real. Treated is not declared. Nobody with the authority to say so has said it, which means the day your pipeline picks one, you have declared it for them, silently, and the other two copies keep being edited by the people who think theirs is the one.

The same logic runs through every box. The vendor API is a fact about the vendor, not about Harbourline: it exists and nobody has ever used it, so it is a possibility, not a source. The loyalty data sits with the app vendor and an export costs extra and needs a contract clause (g12: "Owned means who can say yes to an export. That is usually a contract, not a login."). The bookkeeper's one export a month is in her contract. Each of those is a data-state fact with a signature on it, and each sets a ceiling on your pipeline that no schedule can lift.