The threshold dial

You often need precision and recall folded into one number - to sort candidate models, to put in a dashboard cell. F1 is the harmonic mean: F1 = 2PR / (P + R). The choice of mean is the entire point:

• Take a degenerate model: precision 1.0, recall 0.02 (it flags two sessions all year, both correct). The arithmetic mean says 0.51 - "roughly half good." Flattery.
• The harmonic mean says ~0.039 - roughly as useless as the model actually is.

Harmonic means punish lopsidedness: the score is dragged toward the WORSE of the two numbers, and a zero on either side gives F1 = 0. You cannot buy a good F1 by maxing one metric and abandoning the other - which is exactly the gaming an arithmetic mean would invite. On Lumen's slice at the default 0.50 cut: precision 21.5%, recall 63.6%, F1 0.321 - pulled down toward the weak precision, as it should be.

When you KNOW one error hurts more, F-beta tilts the harmonic mean. Beta scales how much recall counts relative to precision:

• beta = 2 - recall matters more (a miss is the expensive error: fraud, disease, Lumen's $74 lost orders).
• beta = 0.5 - precision matters more (a false alarm is the expensive error: spam filters, auto-takedowns).
• beta = 1 is plain F1.

from sklearn.metrics import fbeta_score, balanced_accuracy_score

fbeta_score(y_test, y_pred, beta=2)    # recall-weighted: misses hurt more
fbeta_score(y_test, y_pred, beta=0.5)  # precision-weighted: false alarms hurt more
balanced_accuracy_score(y_test, y_pred)  # macro-average recall - the honest
# accuracy replacement on imbalanced data (b1's paradox, patched)

An honest flag before you fall in love with it: F-beta is a soft version of what b5 does properly. Beta encodes "recall matters more-ish"; a cost matrix encodes "a miss costs $74 and a false alarm costs $2," and then the optimal threshold is a calculation, not a vibe. Use F-beta when you know the direction of the asymmetry but not its price. Once you have prices, graduate.

balanced_accuracy_score is the macro-average of per-class recall: recall on the converters averaged with recall on the non-converters. The lazy "always predict no" model from b1 scores 96.8% on plain accuracy but exactly 0.5 on balanced accuracy - the same as coin-flipping. That single property is why sklearn's docs recommend it to avoid "inflated performance estimates on imbalanced datasets." It is the drop-in replacement when a stakeholder insists on "an accuracy number."

sklearn hands you the whole menu in one call - every precision/recall pair the model can reach, plus the thresholds that reach them:

from sklearn.metrics import precision_recall_curve

scores = model.predict_proba(X_test)[:, 1]
prec, rec, thresholds = precision_recall_curve(y_test, scores)

# GOTCHA: thresholds is ONE SHORTER than prec and rec.
# The final (precision=1, recall=0) point is appended with no
# threshold attached - so zip against thresholds, not the metrics:
for p, r, t in zip(prec, rec, thresholds):
    print(f"cut at {t:.2f} -> precision {p:.2f}, recall {r:.2f}")

# picking a point: e.g. the highest-recall cut with precision >= 0.30
import numpy as np
ok = np.where(prec[:-1] >= 0.30)[0]
best_t = thresholds[ok[0]]

The length mismatch is not trivia - off-by-one it and every threshold you report is shifted one row from the precision/recall pair you think it produces. You would ship a cut believing it clears a precision bar it does not clear. Zip against thresholds and the alignment is guaranteed.

Ng's fuller version, from Structuring ML Projects: when several things matter, make ONE the optimizing metric and turn the rest into constraints - "maximize F2, subject to latency under 300ms." You will meet this again in b8, where latency and cost per query become first-class satisficing metrics for LLM systems, straight from Anthropic's own success-criteria list.