learn-data-heuristics-with-phoebe / Session 2 of 9
Learn Data Heuristics with Phoebe · Session 2 of 9

The decision it serves

Session 1 measured that six of Tideline's twenty defects are invisible to every interface principle in the canon. Tonight is the first of them: nobody can say what decision the screen exists to support. You will learn the one method built specifically to derive metrics from a decision rather than picking them off a menu - Goal Question Metric - and the one that turns five vague category names into a real selection process - HEART. Then you write Tideline's actual decision statement and add it to the review instrument.

🔴 Advanced Data and analytics leads Anyone who commissions a dashboard 45 min
0-3 · Welcome 3-23 · The decision, named 23-40 · Build-along: Tideline's decision statement 40-45 · Q&A
Part 0

Where we are

Session 1's bench turned on nine lenses one at a time and watched the catch count climb from 5 to 20 out of twenty planted defects. Three of the nine lenses carried six defects that no interface-trained set could see at all, because they are not properties of the interface. This session takes the first of those six: D18, no decision named for the screen, which turns out to drag a second one with it, D9, every panel carrying identical visual weight. Sessions 3 through 5 take the other four.

Live = presented in the 45 minutes Self-study = the full depth, after class 🔍 Every diagram zooms

★ What you walk out with tonight. A real decision statement for Tideline, naming the choice and who makes it. A GQM derivation tree you built, not copied. One HEART category with an actual signal behind its metric. And the exact attribute the review instrument checks for a named decision, added to your own screen.

LiveWhy a dashboard is not a product until something decides on it3 min▶

A screen that nobody can name a decision for is not wrong, it is untestable. There is no way to ask whether it works, because working would mean helping someone choose, and nobody has said what the choice is. That is a different failure from a bad chart. A bad chart still serves a decision badly. A screen with no named decision cannot be evaluated at all - it can only be liked or ignored.

This is also why D18 is not primarily a design defect. It sits upstream of design. Fixing the chart types on a screen that serves no named decision is polishing a door with no room behind it.

Real world

Tideline's retention dashboard is reviewed in a Monday standup that has run for two years. Ask the room what decision the screen changes and you get three different answers, none of them written down anywhere, and none of them matching what actually happens after the meeting.

Self-studyThe vocabulary for today4 min▶
  • Decision statement - the choice being made, who makes it, and how often. Three parts, all three required.
  • Goal - in GQM, the outcome the decision-maker is actually trying to reach, stated once, in words a non-analyst would recognise.
  • Question - something that must be answered to know whether the goal is being met. A goal usually opens two or three of these.
  • Metric - the number that answers one specific question. In GQM a metric with no question above it does not belong on the screen.
  • Signal - in HEART, the observable behaviour or self-report that stands in for a category before it becomes a metric. The step everyone skips.
Part 1 · covers D18 and D9

The named decision, and the dashboard nobody opens

6 min live

A decision statement names three things: the choice on the table, the person or team who actually makes that choice, and how often the choice recurs. Drop any one of the three and the screen drifts back toward decoration. Miss the decision-maker and everyone assumes someone else is using it. Miss the cadence and nobody knows if last week's number is even still relevant.

D18 No decision named for the screen D9 Every panel carries equal visual weight No entry point for the eye to start reading Decision made off screen - gut feel, memory Dashboard nobody opens The dashboard is not badly designed. It was never told what it was for.
🔍 Click to zoom - D18 and D9 are one causal chain, not two unrelated findings
LiveThe three parts of a decision statement3 min▶

Tideline's real decision, the one this session's build-along writes out in full: which at-risk enterprise accounts get a human intervention this week, and who makes that call. Read it against the three parts:

  • The choice - which accounts get a human intervention. Not "how is retention doing", which is a topic, not a choice.
  • The decision-maker - the customer-success team lead who triages the week's list, not "the team" in general.
  • The cadence - weekly, on the Monday triage. A decision with no stated cadence cannot be judged late or on time.

Once the choice is named, D9 has an answer too: the panel that shows which accounts crossed the risk threshold this week is the one that should look different from the other twelve. Right now nothing does, because nothing was ever told it mattered more.

Self-studyTopics are not decisions3 min▶

The commonest failure mode is a screen built around a topic - "churn", "engagement", "growth" - rather than a choice. A topic dashboard can grow forever, because every metric that is vaguely about the topic has a plausible claim to a panel. A decision dashboard has a natural ceiling: once every question the decision-maker needs answered has a metric, adding another panel makes the screen slower to read without making the decision any easier to make.

The test is blunt and worth using out loud in a review: point at any panel and ask "which part of the decision does this change." A panel nobody can answer for is a candidate for deletion, regardless of how interesting the chart is.

Part 2 · covers GQM

Goal, question, metric: derive the numbers, don't pick them

7 min live

Goal Question Metric comes from software measurement, not dashboards: Basili, Caldiera and Rombach set it out in the Encyclopedia of Software Engineering in 1994. The method is a strict order of operations. State the goal first, in the decision-maker's own words. Derive the questions that must be answered to know whether the goal is being met. Only then attach a metric to each question. A metric proposed before its question exists has nothing to justify its place on the screen.

Goal Which at-risk enterprise accounts get a human call this week, decided by whom Q1 Which accounts are trending toward churn this week? Q2 Which of those are worth an intervention? Q3 Who has the authority and time to act this week? Risk-score delta, week over week ARR at risk vs renewal date CSM caseload and owner assigned Read it bottom to top: each metric answers one question, one goal ties all three.
🔍 Click to zoom - the tree derived live from Tideline's own decision
LiveWorking it on Tideline's decision4 min▶

The goal in the figure above is not an abstraction, it is the exact decision statement from Part 1, restated so it can be questioned. Three questions fall out of it almost mechanically once the goal is stated that precisely: which accounts are drifting, which of the drifting accounts are worth the CSM's time, and who actually has the bandwidth to act this week. Each question gets exactly one metric, and each metric has a reason to exist that a reader can trace all the way back to the goal.

Notice what is missing on purpose: a metric like "total signups this month" never appears, because no question above needs it. GQM's discipline is negative as much as positive - it tells you what to leave off the screen at least as often as what to add.

Self-studyWhy GQM resists dashboard sprawl3 min▶

Most dashboard growth happens metric-first: someone has a number they can compute, so a panel gets added for it, and a question gets invented afterward to justify it if anyone asks. GQM inverts the order and makes that inversion visible, because a metric with no upstream question is easy to spot on the tree - it is a box with nothing pointing to it.

This is also the cleanest way to retire a panel without a fight. Instead of arguing that a chart is ugly or unused, trace it up the tree. If it does not answer a live question, and the question does not serve the stated goal, the case makes itself.

Part 3 · covers HEART

HEART's real contribution is the selection step, not the five names

7 min live

Rodden, Hutchinson and Fu introduced HEART at CHI 2010: Happiness, Engagement, Adoption, Retention, Task Success. Everyone remembers the five names. Almost nobody repeats the process the paper actually argues for, which is Goals-Signals-Metrics: for each category, name the goal first, then the observable signal that would tell you the goal is met, and only then the metric that operationalises the signal. Skip the middle step and HEART degrades into five more headings on a dashboard.

Happiness Engagement Adoption Retention Task success Goal Feel confident deciding? Return to the tool daily? New CSMs start using it? Keep using it past week one? Decide who to call, fast? Signal Self-rated confidence Session frequency First decision logged No signal chosen Time to decide, rework rate Metric Weekly survey score Logins per CSM per week Days to first logged use No usable metric Median minutes to decision Skip the signal and the category still gets a box on the dashboard - just not a usable one.
🔍 Click to zoom - a category with no signal produces no usable metric, not a placeholder one
LiveApplying HEART to the decision screen itself4 min▶

HEART is usually applied to a product's end users. It applies just as well to the decision-support screen a team builds for itself, and that is the angle the build-along uses. For a screen meant to help a CSM lead decide who gets called this week, Task Success is the category that matters most: can the decision actually get made, and made quickly.

The goal: a CSM lead can decide who to call without extra digging. The signal: how long it takes from opening the screen to logging a decision, and how often that decision gets reopened afterward. The metric: median minutes from open to logged decision, tracked weekly. Notice the signal came before the metric, not after.

Self-studyThree frameworks that get cited as numbered lists, and are not4 min▶

Three ideas sit right next to GQM and HEART in most people's mental map of decision-and-metric frameworks, and all three get flattened into "the N principles of X" in casual use. None of the three actually ships one.

Commonly cited asWhat the source actually contains
The North Star framework's principlesAmplitude's framework, with the metric concept credited to Sean Ellis: one north-star metric plus a small set of input metrics. A metric-selection method, not a principle set.
Decision intelligence's numbered rulesCassie Kozyrkov defines decision intelligence as an interdisciplinary field spanning statistics, economics and behavioural science, and separates decisions worth automating from decisions that stay human. There is no numbered framework to quote.
The AI hierarchy of needs, with N layersMonica Rogati's June 2017 Hacker Noon argument holds: build the data infrastructure before reaching for AI. Published renderings of her pyramid disagree on whether it has four layers or six.

Never quote a layer count for Rogati's pyramid, and never present the North Star framework or decision intelligence as a numbered set of principles. All three are real, useful, and worth citing by name and argument - just not by a count nobody published.

Build-along

Write Tideline's decision statement

★ 17 min · everyone builds

Work through these in order. Each step produces one artifact you carry into session 9's finished instrument.

Write the decision statement in full: which at-risk enterprise accounts get a human intervention this week, and who makes that call. Name all three parts out loud - the choice, the decision-maker, the cadence - before writing anything else down.

Derive two questions from that goal using GQM. Use the tree above if you need a start: something must identify which accounts are drifting, and something must identify who has the capacity to act this week.

Attach one metric to each question. Check both against the goal directly: if you removed this metric, would the decision-maker be less able to make the call? If the honest answer is no, the metric does not belong yet.

Pick one HEART category for the decision screen itself, not for Tideline's end customers. Name its signal before its metric. Task Success is the strongest fit here, but defend your own choice if you land somewhere else.

Add the data-decision row to your review instrument. The bench that runs in session 9 checks for exactly this: a data-decision attribute on the screen's container, carrying the decision statement as its value. No attribute, no named decision, D18 fires.

Read the statement back as if you were the CSM lead seeing the screen cold. If it takes more than one sentence to say what you are supposed to decide from it, the statement is still a topic wearing a decision's clothes.

Naming the decision does not fix D9 by itself - the panel that answers the goal's first question still needs to look different from the other twelve, and that is a session 6 fix. What naming the decision does is give you a defensible reason to make that panel bigger, which is more than "it felt important" ever bought you in a design review.

Homework

Before session 3

20 min
Check yourself

Three questions

What three things does a decision statement need to name?

All three parts are required. Drop the decision-maker and everyone assumes someone else is using the screen; drop the cadence and nobody can say whether a number is stale.

In Goal Question Metric, what has to come first?

GQM's discipline is the order of operations. A metric proposed before its question exists has nothing above it to justify its place on the screen.

According to the 2010 paper that introduced it, what is HEART's actual contribution?

The five names are what everyone remembers, but the paper's working method is Goals-Signals-Metrics. Skip the signal step and a category still gets a box on the dashboard - it just does not produce a usable metric, as the Retention column in the figure shows.

Sources covered

What this session teaches, and where it came from

✓Basili, Caldiera, Rombach, Goal Question Metric Paradigm, Encyclopedia of Software Engineering vol. 1, Wiley 1994, pp. 528-532Primary verified. The derivation order taught in full and worked on Tideline's own decision
✓Rodden, Hutchinson, Fu, Goals-Signals-Metrics: A Practical Approach to Measuring UX, CHI 2010Primary verified. The five categories and the selection process, applied to the decision screen itself
◐North Star Framework, Amplitude, metric concept credited to Sean EllisReal and useful, but no numbered principle set exists - taught as a metric-selection method only
◐Cassie Kozyrkov, decision intelligenceAn interdisciplinary field with an automatable-vs-human split, never a numbered framework
◐Monica Rogati, AI Hierarchy of Needs, Hacker Noon, June 2017The argument holds; the layer count is disputed across renderings, so none is given

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 2 cheat sheet · pin this

Decision statementChoice, plus decision-maker, plus cadence. All three, or it is still a topic.
GQM orderGoal, then questions, then metrics. A metric with no question above it does not belong.
HEART's real moveGoals-Signals-Metrics selects one signal per category before naming a metric - not the five names alone.
No signal, no metricA HEART category with no chosen signal still gets a box on the dashboard. It just is not usable.
Three real ideas, no countsNorth Star, decision intelligence, and Rogati's hierarchy are genuine - never cite a principle count or a layer count for any of them.
D18 and D9Name the decision with a data-decision attribute, or D18 fires - and D9 usually fires right behind it.