Pay gap decomposition on 11,301 employers

The 2024 file holds 11,315 filings, one per employer. Two cuts, each written as a decision before any number. Kept: all eight quartile shares and both hourly gaps present, and a mean gap within -100 to 100; that leaves 11,312, because three filings report a mean gap below -100, which would mean women are paid more than twice what men are on average and is almost certainly a keying error. Usable: at least one man and one woman across the four bands, because an all-male or all-female workforce has no gap to decompose and would divide by zero in the next Part; that leaves 11,301. The 2023 file gives 11,169 filings and 11,162 usable on the same rules.

On the 11,301, the reported median hourly gap has a median of 8.4 percent (8.9 in 2023) and a mean of 11.2. 77.6 percent of employers report a positive median gap, men paid more, and 8.5 percent report exactly zero. The reported mean gap has a median of 11.0 and a mean of 12.0, with 85.9 percent positive. Those are medians across employers, and every one of them counts a small employer and a very large one the same.

Then the pyramid: female share by band at the median employer, 56.0 / 52.7 / 47.0 / 40.0 percent bottom to top, and 81.3 percent of employers have a higher female share in the bottom band than in the top. That is not a statement about any one employer. It is the shape of the UK filings, and it is the only pay structure the filing lets anyone see.

The filing says, for one snapshot date, how hourly pay differs between men and women on average and at the middle, how bonuses differ, and where each sex sits among four bands of equal headcount. It says nothing about pay within a band, nothing about hours, occupation, tenure or part-time status, and nothing about why. A median gap of exactly zero, reported by 8.5 percent of employers, is a statement that the middle man and the middle woman fall on the same hourly rate; it can happen when many people share one hourly rate, and it says nothing about the other 49 percent on each side.

The bonus figures are thinner still: 75.8 percent of filings report a median bonus gap, and the rest pay no bonus to one sex or to either. Tonight uses the hourly gaps and the quartiles only. Deep-dive session 6 looks inside a band with a regression on constructed salaries, which is exactly what the filing cannot do.

Because the four bands are equal in headcount, the number of women in a band is proportional to the female share of that band. So women's average pay under a ladder L is the ladder weighted by the four female shares, men's is the ladder weighted by the four male shares, and the composition-only gap is 1 minus women's weighted mean over men's weighted mean. It is the gap an employer would report if pay differed only by band and everyone in a band were paid the same. Equal pay inside every band is the assumption, and it is stated wherever the number appears.

The fit tries every ladder with the bottom band fixed at 1 and each rung above the last, in steps of 0.05, and keeps the one whose composition-only gaps sit closest to the reported mean gaps in squared error across all 11,301 employers. The winner is 1 / 1.20 / 1.45 / 2.80. Under it, composition alone explains R2 0.353 of the variance in reported mean gaps, a correlation of 0.66. About a third of the variation in reported mean gaps across employers is accounted for by the shape of the pyramid under one national ladder.

What the ladder is not: it is not any employer's pay scale, and it is not a measurement of what the top band earns. It is the single set of four numbers that best turns 11,301 pyramids into 11,301 reported gaps. An employer whose own top band pays three times its bottom will have a steeper ladder than this one, and its composition-only gap under the national ladder will be too small. That is a limitation to print, not a defect to hide.

Take the median employer's band profile, whose composition-only gap under the fitted ladder is 9.57. Now hire women equal to 10 percent of headcount, all into one band. Put them in the bottom band and the gap becomes 14.88: wider, because the new women all sit on the lowest rung and pull women's average pay down. Put the same number of women in the top band and it becomes -3.18: reversed. Both are modelled on computed numbers under the fitted ladder and equal pay within a band.

The lesson is the one every pay gap report should carry: "hire more women" is not a lever; "hire more women where" is. A target stated as a headcount can widen the reported gap while the organisation becomes more balanced overall, and a board that reads only the gap will conclude the programme failed. The build-along's figure draws the two arrows, and deep-dive session 10 lets you pull the lever on 3,000 real employers in the browser.

A general-purpose optimiser would find a ladder faster, and it might find a slightly different one each time depending on its starting point and tolerance. The grid is three nested loops over values in steps of 0.05, with each rung forced to be at least 0.05 above the last, and it returns the same four numbers on every machine. For a number that will be quoted in a report, reproducible beats fast; the fit on 11,301 employers takes seconds.

The constraint matters too. An unconstrained fit could return a ladder in which the upper middle band pays less than the lower middle, which fits some noise and describes no pay structure anyone has. Forcing the rungs to rise is a decision about what a pay quartile is, and it goes in the decisions block with the others. On the bench's 3,000-employer sample the same grid gives 1 / 1.3 / 1.5 / 3.0 with R2 0.4114: a different sample, a similar ladder, which is itself a small robustness check.

At the median, the pyramid under the national ladder produces most of the reported gap: 9.8 percent against 11.0. The residual, reported minus composition-only, has a median of 2.3 points. Read carefully, that says: for the middle employer, the gap it reports is a little more than its band shares alone would produce if everyone in a band were paid the same and its bands were spaced like the nation's. The spread is wide, a standard deviation of 10.2 points, which means the median hides employers far above and far below the line.

33.8 percent of employers report a gap below their composition-only gap. At those employers either pay inside the bands runs the other way, women paid more than men within a band, or the national ladder is steeper than their own. The filing cannot tell those two apart. That is the honest limit of a decomposition built on four bands: it separates the part of the gap that the pyramid would produce from the part it would not, and it cannot say what the second part is.

The sentence to write: "under a ladder fitted to all 2024 filings, and assuming equal pay within bands, our band profile alone would produce a mean gap of X; we report Y." Every clause is a stated assumption, and none of them is a cause.

A decomposition that explains a third of the variance could be explaining it for the wrong reason. Perhaps any four numbers between 0 and 1 would correlate with the gap; perhaps it is the overall female share, not the pyramid, doing the work. The test: take each employer's four female shares and shuffle them among its own four bands, so each employer keeps exactly the same shares but loses their order, and refit under the same ladder.

Done 100 times on the bench's 3,000-employer sample (drawn with numpy.random.default_rng(20240405)), the R2 averages about -1.84, standard deviation 0.07, best draw -1.68, against 0.408 for the same sample with the bands in their real order. A negative R2 means the prediction is worse than guessing the average gap for everyone. Composition explains the gap only when the bands are in their real order, which is the evidence that the pyramid, and not the share of women, is the mechanism. Any explanation you publish deserves a test that could have broken it.

Tonight's split is a four-band, population-level version of an older idea. Blinder 1973, "Wage Discrimination: Reduced Form and Structural Estimates", Journal of Human Resources 8(4) 436-455, and Oaxaca 1973, "Male-Female Wage Differentials in Urban Labor Markets", International Economic Review 14(3) 693-709, decompose a gap in average pay into a part "explained by measured differences" in characteristics and a part "not explained". With individual salaries, a regression replaces the ladder and grade, tenure or hours replace the bands. Deep-dive session 6 builds it on constructed salaries.

The warning carries across. Explained is not the same as justified. The composition-only gap says the pyramid would produce 9.8 percent; it does not say the pyramid is fair. Who reaches the top band is itself a set of decisions about promotion, hiring and hours, and a gap that is "explained by composition" is a gap whose cause has been moved one step up the chain, not removed.

The Office for National Statistics reports the UK gender pay gap from the Annual Survey of Hours and Earnings, April 2024: 7.0 percent for full-time employees and 13.1 percent for all employees (2023: 7.5 and 14.2). Those are gaps between hourly-earnings medians across the whole economy, one median for all men and one for all women. The 8.4 above is a median across employers of each employer's own median gap, where a firm of 260 and a firm of 26,000 count once each.

The two are not comparable, and neither is wrong. The ONS number answers "how does the middle woman's pay compare with the middle man's, nationally"; the filing median answers "what does the typical large employer report". Put them in one sentence only with both definitions attached.