learn-people-analytics-with-phoebe / Foundations session 3 of 6
Learn People Analytics with Phoebe · Foundations session 3 of 6

The hiring funnel

Two sessions on the drain; tonight the tap. A vacancy attracts applicants, a screen keeps some, interviews keep fewer, offers go to a handful and most are accepted. Five stages, and between each pair a ratio, and every ratio has a base that is the stage before it. That is the whole grammar of a hiring funnel, and it is broken in the same two ways every time: a ratio quoted without saying what it is a ratio of, and two clocks called by each other's names. Apex's funnel is constructed, 400 applicants to 10 hires, invented to be worked by hand. The last Part adds one metric that every funnel should carry and most do not: whether the screen keeps groups at the same rate.

🟢 Foundations Leaders, HR partners, hiring managers, anyone who reads a recruiting report No code 45 min
0-3 · Welcome 3-18 · Stages, ratios, two clocks, one fairness metric 18-40 · Build-along: the Apex funnel worksheet 40-45 · Q&A
Part 0

Where we start

Recruiting reports are the most ratio-dense pages in any people pack, and the ratios are the least labelled. "Offer acceptance is 83 percent" and "we hire 2.5 percent of applicants" can both be true of the same funnel, and a reader who does not know the base of each will conclude the funnel is both healthy and broken. Foundations sessions 1 and 2 built the habit of writing the base under a rate; tonight applies it to a funnel, where the base changes at every step, then to time, where two different clocks share one vacancy, and finally to fairness, where the metric is a ratio of two rates and both rates have to be shown.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = everyone works it through

★ What you walk out with tonight. A funnel with a base under every ratio, the difference between a stage ratio and a yield, two clock definitions you can hold a recruiter to, and one number, the impact ratio, that tells you whether a screening stage is keeping groups at the same rate, with the evidence from two studies on why that stage deserves the scrutiny.

Part 1 · covers stages, ratios and their bases

Every stage is a rate, and its base is the stage before

6 min live

Apex opened ten roles and ran one funnel for them, constructed for tonight: 400 applicants, 120 passed the screen, 40 were interviewed, 12 received offers, 10 accepted. Between each pair of stages sits a ratio, and each ratio's base is the stage before it: 120 of 400, 40 of 120, 12 of 40, 10 of 12. Four ratios, four bases, and the base has to be said aloud because "30 percent" is true of two of them.

FIVE STAGES, FOUR RATIOS, FOUR BASES Apex, ten roles, constructed · widths not to scale · each base is the band above 400 applicants 120 screened in 40 interviewed 12 offers 10 hires 120 of 400 = 30.0% screened, of applied 40 of 120 = 33.3% interviewed, of screened 12 of 40 = 30.0% offered, of interviewed 10 of 12 = 83.3% accepted, of offered overall yield: 10 of 400 = 2.5% two of the four ratios read 30.0 and mean different things A stage ratio's base is the stage before, so 30.0 describes the screen and the offer alike. Print the base.
🔍 Click to zoom - the moss band is what the whole funnel was for; the four ratios beside it are each a fraction of the band above
LiveFour ratios, and what each one is a ratio of4 min▶

Screen-in rate: 120 of 400, 30.0 percent. The base is everyone who applied. This is the ratio the rest of tonight cares most about, because the screen is the stage with the most people and the least time per person, and it is where the two studies in Part 3 found the trouble.

Interview rate: 40 of 120, 33.3 percent. The base is the screened, not the applicants. Quote it as "a third" and someone will hear "a third of applicants get an interview", which would be 133 people, not 40.

Offer rate: 12 of 40, 30.0 percent. The base is the interviewed. It reads the same as the screen-in rate and describes a stage with a tenth of the people in it. Acceptance rate: 10 of 12, 83.3 percent. The base is offers made, and it is the one ratio a company reads as a verdict on itself: two people said no.

All four are constructed for Apex, and the arithmetic is the point, not the values. The rule is the same as for attrition: the base goes beside the ratio, every time, and the base of a funnel ratio is the stage before it unless somebody says otherwise. The overall yield, 10 of 400, is the one ratio whose base is the top of the funnel, and it is the number a hiring manager should hear first: 400 applications bought 10 people.

LiveStage ratio or yield: the two readings of one funnel2 min▶

Every stage can be read two ways. Against the stage before, it is a conversion: how selective was this step. Against the top of the funnel, it is a yield: how much of the original 400 is still here. At the interview stage, Apex reads 33.3 percent one way and 10.0 percent the other (40 of 400). Neither is wrong, and a report that mixes them, a conversion on one line and a yield on the next, will be read as a funnel that leaks 23 points between two rows that describe the same forty people. The build-along draws both columns side by side so that the difference is visible rather than argued.

Real world

"Our offer acceptance is fine, so the funnel is fine." A recruiting update leads with 83.3 percent acceptance and the room relaxes. Ten of twelve offers accepted is genuinely good. It is also the last and smallest stage, and it says nothing about the 280 people the screen rejected or the 80 the interview did. A funnel is judged at its widest stage, not its narrowest, because that is where the decisions are cheapest to make and most numerous. Constructed example; the habit is universal.

Part 2 · covers the two clocks

Time-to-fill and time-to-hire are two clocks

4 min live

Ask how long hiring takes and two numbers come back with different names and, often, the wrong name on each. Time-to-fill starts when the requisition is opened and stops when an offer is accepted: it measures the vacancy, from the company's side. Time-to-hire starts when the person who was eventually hired entered the funnel and stops at the same acceptance: it measures the candidate's journey. Same vacancy, same end, two different starts, and fill is always the longer of the two.

ONE VACANCY, TWO CLOCKS the same end, two different starts · no day counts, the definitions are the point requisition opened job posted the hire applies interviews offer accepted first day TIME TO FILL: requisition opened to offer accepted, the vacancy's clock TIME TO HIRE: the hire applies to offer accepted, the candidate's clock the person who was eventually hired Fill is always the longer. Swap the names and the company looks faster than it is, or the candidate's journey looks slower. A third clock, time to productivity, has no agreed start or end, so it rarely compares.
🔍 Click to zoom - the moss bracket starts when the company decided to hire; the hatched one starts when the right person turned up
LiveWhich clock, and what each one is for3 min▶

Time-to-fill measures the vacancy: from the day the requisition was opened, or approved, to the day an offer was accepted. It is the number a hiring manager feels, because it is how long the seat was empty, and it includes the days before anyone applied. It is driven by approval speed, posting speed, sourcing and every stage of the funnel.

Time-to-hire measures the candidate: from the day the person who was eventually hired entered the funnel to the day they accepted. It is the number the candidate feels, and it is the better measure of how fast the process itself runs once there is somebody in it. It excludes the days the requisition sat waiting.

Both stop at acceptance, not at the first day, because the notice period belongs to the candidate's old employer. Both definitions are standard practice with no single source, and both drift: some teams start fill at posting rather than approval, some start hire at first contact rather than application. The only defence is to write the start and end events beside the number, exactly as the base goes beside a rate. A time-to-hire quoted without its start event is a number without a base.

Self-studyAverages of clocks, and the vacancies that never closed2 min▶

An average time-to-fill is computed over vacancies that were filled. The ones still open at the reporting date are not in it, and they are, by construction, the slow ones. This is the survivor problem from Foundations session 1 wearing a different hat: the average describes the vacancies that closed fast enough to be counted. The median is less distorted than the mean, and the honest report says how many requisitions were still open on the date and how old the oldest was. Deep-dive session 2 measures the same effect on tenure, where it is called censoring, and the fix is the same: count the ones that have not finished yet, and say so.

Real world

Two teams, two clocks, one slide. One recruiting team reports time-to-hire from the candidate's application; another reports time-to-fill from approval, and both call it "time to hire". The first team looks twice as fast. The difference is not performance, it is the start event, and the slide that compares them is comparing a stopwatch with a calendar. Constructed scenario; the naming drift is everywhere.

Part 3 · covers adverse impact as one metric

Does the screen keep groups at the same rate

5 min live

A funnel ratio has a base, and the base can be split. Split the screen-in rate by sex, by ethnicity or by age band, and each group has its own selection rate at that stage. The impact ratio is the lower rate over the higher. It is one number, it is a ratio of two rates, and both rates with both bases belong beside it. That is the entire metric. Everything after it, the thresholds, the simulator, the legal map, lives in the AI + HR course and is linked, not repeated.

THE IMPACT RATIO IS TWO SELECTION RATES Apex screening stage, split by sex, invented · bar height is people, 0.6px each MEN, invented split 240 80 applied screened in 80 of 240 = 33.3% WOMEN, invented split 160 40 applied screened in 40 of 160 = 25.0% IMPACT RATIO 25.0, the lower rate 33.3, the higher rate = 0.75 a ratio of two rates, so both rates and both bases go on the slide beside it One stage, one split, one metric. Whether 0.75 is a problem has a rule of thumb, a simulator and a legal map behind it, all in the AI + HR course. Here the metric is enough: compute it at the widest stage.
🔍 Click to zoom - the wine bars are the people the screen kept; the ratio is the shorter wine bar's rate over the taller one's
LiveAdverse impact as one metric: the impact ratio at the screen3 min▶

Take Apex's screening stage and split it by sex with an invented split: 240 men applied and 80 were screened in, 160 women applied and 40 were screened in. Two selection rates, 33.3 and 25.0 percent, each with its base. The impact ratio is the lower rate over the higher: 25.0 over 33.3, which is 0.75. It is one number, it is computed at one stage, and it can be computed at every stage and for every group the company records.

The US Uniform Guidelines on Employee Selection Procedures (29 CFR 1607.4(D), 1978) give the rule of thumb that a selection rate under four-fifths of the highest group's rate is generally regarded as evidence of adverse impact. That is the one sentence of law on the page. What counts as a group, what to do when the numbers are small, how a screening tool shifts the ratio, and what the New York City audit law asks employers to publish are all taught with a working simulator in the AI + HR course, and deep-dive session 4 does the arithmetic in pandas.

The habit for tonight is smaller: a funnel report carries the impact ratio at its widest stage, with both rates and both bases printed, the way an attrition rate carries its base. A ratio of two rates reported as a single decimal is two undeclared bases at once.

Self-studyWhy the screening stage earns the scrutiny: Bertrand and Mullainathan 20042 min▶

Bertrand and Mullainathan 2004, "Are Emily and Greg More Employable Than Lakisha and Jamal?", American Economic Review 94(4) 991-1013, sent otherwise comparable resumes to real job advertisements with names chosen to sound white or African-American and counted the callbacks. Resumes with white-sounding names received about 50 percent more callbacks. The stage they measured is the screen: the moment a person reads a resume for a few seconds and decides who goes forward. It is the widest stage, the fastest decision, and the one where the impact ratio in the figure above is computed. That is why the metric belongs at the top of the funnel and not at the offer.

Self-studyDiscretion against the test: Hoffman, Kahn and Li 20182 min▶

Hoffman, Kahn and Li 2018, "Discretion in Hiring", Quarterly Journal of Economics 133(2) 765-800, studied firms that introduced a job test to rank applicants for low-skill service jobs, where managers could still hire against the test's recommendation. Test-based hiring raised completed tenure by about 25 percent (0.23 log points), and the hires managers made by overriding the test left sooner than the ones the test would have chosen. The study does not say managers know nothing; it says that in that setting, the exceptions they made were on average worse than the rule they overrode. For a funnel it means the screening stage is measurable, and that "the manager's judgement" is a stage with a selection rate like any other, which can be compared with the alternative it replaced. Deep-dive session 9 returns to the same paper when the screening tool is an algorithm.

Build-along

The Apex funnel worksheet, on paper

★ 22 min · everyone works it through

Six worksheet steps that turn a recruiting update into a funnel a reader can check: the stages with counts, the two ratio columns with their two different bases, the two clocks with their start and end events named, the impact ratio at the widest stage with both rates shown, and the sentence. Apex's funnel is invented; deep-dive session 4 rebuilds it in pandas with the same labels. Work each step for a funnel you know.

THE SAME FUNNEL, TWO BASES Apex, constructed · bars 3px per point · left column base is the row above, right column base is 400 STAGE N OF THE STAGE BEFORE OF THE 400 APPLICANTS applied 400 the top of the funnel 400 of 400 screened in 120 30.0% 30.0% interviewed 40 33.3% 10.0% offered 12 30.0% 3.0% hired 10 83.3% 2.5% The forty interviewed people are 33.3 percent in one column and 10.0 in the other. Same people, different base. Left column: where the funnel narrows. Right column: what 400 applications bought. Report both, labelled.
🔍 Click to zoom - the moss sliver is the whole point of the funnel, and it is invisible on a chart that only shows stage ratios

Step 1 · Name the stages and count them. Five stages is usual; yours may have six. Each count is a stock of people who reached that stage during the period. Write the period. Apex: ten roles, one funnel, constructed.

★ Step 1 · stages and countsapplied 400 (Apex, constructed) screened in 120 interviewed 40 offered 12 hired 10 yours: _________ ____ _________ ____ _________ ____ _________ ____ _________ ____

Step 2 · The stage ratios, base = the row above. Four ratios for five stages. Write each as "n of base", not as a percent alone.

★ Step 2 · of the stage beforescreen-in 120 of 400 = 30.0 percent interview 40 of 120 = 33.3 percent offer 12 of 40 = 30.0 percent acceptance 10 of 12 = 83.3 percent yours: ___ of ___ = ____ ___ of ___ = ____ ___ of ___ = ____ ___ of ___ = ____

Step 3 · The yields, base = the top. The same stages against the 400. This is the column a hiring manager should see first, and the one most reports leave out.

★ Step 3 · of the applicantsscreened in 120 of 400 = 30.0 percent interviewed 40 of 400 = 10.0 percent offered 12 of 400 = 3.0 percent hired 10 of 400 = 2.5 percent (400 applications bought 10 people) yours: ___ of ___ = ____ ...

Step 4 · The two clocks, with their events. Write the start event and the end event beside each clock. If your system uses different events, write the ones it uses; the point is that they are written.

★ Step 4 · two clocks, four eventstime to fill: from requisition opened to offer accepted = ____ days (median, filled vacancies only) time to hire: from the hire's application to offer accepted = ____ days (median) still open on the reporting date: ____ requisitions, oldest ____ days

Step 5 · The impact ratio at the widest stage. Split the screen-in rate by one group the company records. Both rates, both bases, then the lower over the higher. Apex's split is invented.

★ Step 5 · one metric, two rates, two basesmen 80 of 240 = 33.3 percent (invented split) women 40 of 160 = 25.0 percent impact ratio = 25.0 / 33.3 = 0.75 yours: group A ___ of ___ = ____ group B ___ of ___ = ____ ratio = ____ what the ratio means, and the simulator: the AI + HR course

Step 6 · The sentence. One line with the yield, the narrowest stage, the two clocks with their events, and the impact ratio with its rates. Every number has its base.

★ Step 6 · the sentence"400 applications for ten roles produced 10 hires, 2.5 percent. The screen kept 120 of 400 and the interview 40 of 120; 10 of 12 offers were accepted. Time to fill, requisition opened to acceptance, and time to hire, application to acceptance, are reported separately. At the screen, 80 of 240 men and 40 of 160 women were kept, an impact ratio of 0.75 (split invented)."

Judge the funnel at its widest stage. The screen has the most people, the fastest decisions and the two studies behind it. Acceptance is the stage that flatters; the screen is the stage that matters, and it is where the impact ratio is computed.

Homework

Before Foundations session 4

30 min
  1. Draw a funnel you know with counts and both ratio columns, stage ratios and yields, and mark the one stage where the base is most often left off in your organisation's reports.
  2. Find the two clocks in a recruiting report and write the start event each one actually uses. If the report has one clock with no start event, that is the finding.
  3. Compute one impact ratio at the widest stage for one recorded group, with both rates and both bases printed. If the counts are too small to trust, write that instead of the ratio.
  4. Bring the funnel. Foundations session 4 moves from who gets in to what they are paid: the gender pay gap, on 11,315 real UK filings, and why a gap is not the same thing as unequal pay.
Check yourself

Three questions

Clear all three and Foundations session 3 gets stamped in your passport.

1 · A recruiting report says "interview rate 33 percent" and, two rows down, "interviewed: 10 percent". Apex's funnel is 400 / 120 / 40 / 12 / 10. Which is right?

A stage ratio's base is the row above; a yield's base is the top of the funnel. Both are useful, and a report that mixes them without labels reads as a funnel leaking 23 points between two rows about the same people.

2 · Two recruiting teams both report "time to hire". Team A starts the clock at requisition approval; Team B starts it when the eventual hire applied. Both stop at offer acceptance. Team B looks twice as fast. What is going on?

Same end, two starts. A clock quoted without its start event is a number without a base, and the naming drift between fill and hire is the most common way a hiring speed comparison misleads.

3 · At Apex's screen, 80 of 240 men and 40 of 160 women were kept (an invented split). What is the impact ratio, and what has to appear beside it?

The impact ratio is a ratio of selection rates, lower over higher, computed at one stage. The counts and the applicant shares are its ingredients, not the metric. Computing it at the widest stage is the habit; interpreting it is the neighbouring course's job.

Sources covered

What this session teaches, and where it came from

✓Funnel stages, conversion ratios, yields, time-to-fill and time-to-hireStandard practice, no single source; the two clocks are defined by their start and end events on the page
✓The Apex funnel 400 / 120 / 40 / 12 / 10, the ratios 30.0 / 33.3 / 30.0 / 83.3, the yields 30.0 / 10.0 / 3.0 / 2.5, and the sex split 240 / 80 and 160 / 40 giving an impact ratio of 0.75Constructed; the arithmetic is the lesson, the values are invented
✓29 CFR 1607.4(D), Uniform Guidelines 1978: the impact ratio's rule of thumbPrimary; named once, taught in depth with the simulator in the AI + HR course
✓Bertrand and Mullainathan 2004, "Are Emily and Greg More Employable Than Lakisha and Jamal?", American Economic Review 94(4) 991-1013: about 50 percent more callbacks for white-sounding namesPrimary
✓Hoffman, Kahn and Li 2018, "Discretion in Hiring", Quarterly Journal of Economics 133(2) 765-800: test-based hiring raised completed tenure by about 25 percent (0.23 log points); overrides left soonerPrimary, full text read
◐Averages of clocks over filled vacancies onlyNamed as the survivor problem in another form; measured as censoring in deep-dive session 2

Every fact on this page, and its verification tier, is recorded in the course's source map.

Foundations session 3 cheat sheet · pin this

Base = the stage before120 of 400, 40 of 120, 12 of 40, 10 of 12. Two of them read 30.0 and mean different things.
Yield = base is the top10 of 400 = 2.5 percent. The number a hiring manager should hear first.
Time to fillRequisition opened to offer accepted. The vacancy's clock. Always the longer.
Time to hireThe hire's application to offer accepted. The candidate's clock. Write the start event.
Impact ratioLower selection rate over higher, at the widest stage. 25.0 / 33.3 = 0.75 (invented). Both rates, both bases.
Why the screenAbout 50 percent more callbacks by name (2004); overrides of a test left sooner (2018). Judge the funnel where it is widest.