Survey psychometrics

Take the total score, the sum of the twelve items per person. Its variance has two sources: each item's own variance, and the covariance between items. If the items measure one thing, they co-vary, and the variance of the total is much larger than the sum of the item variances. Alpha is k over k minus 1, times one minus the ratio of those two, so it approaches one when the covariance dominates and falls toward zero when the items are unrelated noise. Four lines of numpy; step 2 of the build-along.

What it does not compute. Alpha does not say the items measure engagement, or anything with a name; it says they measure the same thing as each other. Twelve rewordings of one question give a superb alpha and no information. Alpha also rises mechanically with the number of items, so a long scale with weak items can outscore a short scale with strong ones. Read it with the item-rest correlations in Part 2, never alone, and never as a grade for the survey.

Foundations session 5 drew the same mean from two distributions: a team where everyone answered 3, and a team where half answered 1 and half answered 5. The mean is identical; the teams are not. In code the habit is to print the distribution beside the mean: the share at each point, or at least the share at 1 and 2 together and at 4 and 5 together. A change of 0.2 in a mean can be everyone moving slightly or a few people moving a long way, and only the distribution says which. The same applies to alpha: it summarises a covariance matrix, and the matrix is what to look at when the number surprises you.

The decision to drop an item is not statistical alone. Alpha says the item does not share variance with the rest; it does not say whether the item was wrong or the rest were. If the eleven items ask about the manager and the twelfth asks about pay, the twelfth is a different construct, not a bad question, and it belongs in its own scale. In the constructed survey the case is clean because the generator made it so. In a real survey, read the item's wording before reading its correlation, and write down why it was dropped, because every team's score will move when it goes and someone will ask.

Say what moved. When an item is dropped, the score changes for everyone, by an amount unrelated to anything anybody did. Publish the old score, the new score and the reason, once. A dashboard that silently switches from twelve items to eleven between two quarters has manufactured a trend, and that trend will be read as one.

Step 6 of the build-along correlates the survey score with an invented flag for leaving within six months, and finds a correlation, because the generator planted one. On a real table the same correlation appears often, and it does not say which way the arrow points: disengaged people leave, people who have already decided to leave answer surveys as disengaged, and a bad quarter produces both. The figure "engagement drives performance", in any of its circulating forms, has no measured source in the map and is not claimed. The experimentation and causal inference courses are where an arrow gets a direction; a survey correlation is a reason to design that study, not a substitute for it.

In the constructed survey the trait is known for all 1,022 people, so the mean trait of the 600 who answered can be compared with the mean of the 422 who did not, and the gap is the bias. On a real survey the trait of the non-respondents is exactly what is unknown, and the only handles are the things known for everyone: department, tenure, grade, entity. Compare the respondents to the invited on those. If Sales, with its 50 percent attrition on the start base, answered at half the rate of Finance, the company score is a Finance score wearing a company label, and reweighting by department is the least that is owed.

The threshold is a count, applied first. Group the respondents, count each group, and drop every group under the minimum before computing a single mean. The order matters: a pipeline that computes means and then hides the small ones has already put the means in a file. The minimum itself is a choice; five to ten is common practice with no single source, and the code uses eight and prints the word "stated" beside it. What is not a choice is that the threshold applies to every cut, including the cuts nobody planned: department by tenure band by sex is a small group even when each margin is large.

Gallup's State of the Global Workplace reported 23 percent of employees engaged in its 2024 edition, on 2023 data. The 2025 edition reported 21 percent and the 2026 edition 20. Any of the three is quotable with its edition named; "Gallup says 23 percent" without the year is a number drifting loose from its measurement, and it is reported here as Gallup's figure, not recomputed. The sentence that usually travels with it, that people leave managers and not companies, has no measured source: it traces to Buckingham and Coffman's First, Break All the Rules of 1999, a book and not a statistic, and it is not claimed on any page here. Apex's own exit reasons give Management/Culture 23 of 210, behind Better Opportunity, Career Growth and Compensation, and a reason on an exit form is what the leaver chose to write, which is its own response bias.