Fellegi-Sunter, and the band nobody implements

Two records agree on the surname Smith. Two other records agree on the phone number 0422 909 118. Both are exact agreements, and they are worth wildly different amounts, for a reason that has nothing to do with the strings.

Lots of unrelated people are called Smith, so a surname agreement happens by chance reasonably often. Almost nobody shares a mobile number, so a phone agreement almost never happens by chance. The evidence a field provides depends on how surprising the agreement is, and that is an empirical property of your data rather than a judgement about the field.

Fellegi, I.P. and Sunter, A.B., "A Theory for Record Linkage", Journal of the American Statistical Association, volume 64, number 328, 1969, pages 1183 to 1210. It formalises the approach Newcombe and colleagues published in Science ten years earlier, which session 1 covered.

It is worth reading for one reason beyond the model: it treats linkage as a decision problem with costs, not a classification problem with an accuracy score. The paper's central theorem is about minimising the amount of human review needed to hold two error rates below chosen bounds. That framing is what produces the three-way rule in Part 3, and it is the part that gets lost when the model is reimplemented as "score the pairs and pick a cutoff".

People reach for the field they trust, meaning the field that is usually filled in and usually correct. That instinct optimises m and ignores u, and u is where the discriminating power lives.

Company name in Meridian has an m of 0.95, which is excellent: when two records really are the same person, they almost always agree on the employer. It is also the weakest field in the model, because its u is 0.06 - plenty of unrelated people work at the same company. Agreement is worth only about +4.0. The phone number has a lower m of 0.88, because people change numbers, and yet agreement is worth +8.8, because its u is 0.002.

A field that disagrees is not neutral, it is evidence against the pair, and its size follows the same logic:

• Agreement adds log2(m / u).
• Disagreement adds log2((1 - m) / (1 - u)), which is negative.

Notice what makes a disagreement expensive. Email in Meridian has an m of 0.55, because half of all true pairs have a work address on one record and a personal one on the other, so an email disagreement barely counts against a pair at about -1.2. Company has an m of 0.95, so a company disagreement is a real problem at about -4.2. A field where matches often disagree cannot punish you for disagreeing.

Missing values are a third case, and the honest treatment is to let them carry nothing at all. If one record has no email, that is neither agreement nor disagreement, so the field contributes zero rather than a penalty. The bench does exactly this, which is why Aisha Bello's two records still match on their other fields.

The table in the bench is a stand-in, and the page says so. In real work you get these numbers three ways:

• Count u directly. It needs no labels at all. Take random pairs from your data, which are overwhelmingly non-matches, and count how often each field agrees. For a surname you can do better and use a frequency table, so that agreeing on Smith is worth less than agreeing on Okonkwo. Splink supports exactly this as term-frequency adjustment.
• Estimate m from your labels. Among the pairs you labelled as matches in session 1, count how often each field agrees. Twenty labelled pairs gives a rough but usable number.
• Estimate both with expectation maximisation. Fit the model without labels by treating match status as a latent variable. This is what Splink does by default, and it is why Splink can get started on unlabelled data. It is also why you should still hold labels back to check the result.

The full weight for a pair includes one more term, a prior for how common matches are at all: log2(lambda / (1 - lambda)). In Meridian 35 of 1,953 pairs match, so that term is about -5.7, and it is why a single agreeing field is not enough to call a match. One name agreement is +5.5 against a prior of -5.7, which lands just below zero. Two strangers agreeing on one field is exactly what you would expect to see.

Summing the weights assumes that, once you know whether a pair is a match, the fields agree or disagree independently. Two places that fails, both visible in Meridian:

• Address and postcode. If the street agrees the postcode almost certainly agrees, so counting both double-counts one piece of evidence and inflates the weight. Treat them as one comparison or accept the inflation knowingly.
• Company and email domain. Sarah's email changed because her employer changed. Two disagreements, one underlying cause, counted twice against the pair. Her total of 0.84 is pulled down partly by double-counting a single life event.

The usual remedies are to combine correlated fields into one comparison, or to model the interaction explicitly. Splink lets you do the former easily and the latter with effort. What you should not do is add every column you have and assume more evidence is better; correlated fields make the model overconfident in both directions.

Because those two pairs score the same, a single cut has to get one of them wrong. The bench measures all three options over the whole file:

Both single cuts land on the same F1 of 0.971, and neither can do better, because the error is not a tuning failure. Move the cut down and you gain the true pair and buy a false merge. Move it up and you lose the true pair. That is the trade the band refuses to make.

And the band's price is tiny. Two pairs out of 1,953 go to a person. Those two minutes of human attention recover the real match and reject the father and son, taking the result to 100 percent precision and 100 percent recall - which neither single cut can reach at any threshold.

Fellegi and Sunter derive the thresholds from error bounds rather than taste: you state the false-match rate and the false-non-match rate you are willing to accept, and the thresholds follow. In practice teams work backwards from two constraints instead, which is legitimate as long as it is deliberate:

• The upper threshold is set by the cost of a wrong merge. Un-merging two customers after a year of shared history is expensive and sometimes impossible, so where merges are hard to reverse, set it high.
• The lower threshold is set by review capacity. The band is only useful if somebody actually works it. A band containing 4 percent of a million-record file is 20,000 pairs a night, which nobody reviews, and an unworked queue is worse than a decision because it looks like control.
• Both should be revisited when the data changes. A new source system with a different email convention shifts every email weight in the file.

One honest asymmetry worth stating to stakeholders: a missed match and a false merge are not equal errors. A missed match leaves two customer records where there should be one, which is a known and fixable annoyance. A false merge fuses two real people's data, which may be a privacy incident and is much harder to unpick. Session 6 returns to this.