Precision, recall and F1

The trade-off you’ll be asked to reason about in almost every ML interview, usually framed as a business scenario rather than a formula.

The trade-off is the threshold

A classifier outputs a score. Precision and recall are properties of a threshold, not of the model.

scores = model.predict_proba(X_val)[:, 1]
pred = scores >= 0.5           # 0.5 is a default, not a decision

• Raise the threshold → fewer positive predictions → precision up, recall down.
• Lower it → more positive predictions → recall up, precision down.

So “improve precision” and “improve recall” are usually threshold moves, not model changes. Only a genuinely better model raises the whole curve. Separating those two levers in your answer is the main thing being tested.

F1 and F-beta

The harmonic mean of precision and recall:

F1 = 2 * P * R / (P + R)

Harmonic, not arithmetic, so it punishes imbalance between the two. Precision 1.0 with recall 0.01 gives F1 ≈ 0.02, whereas the arithmetic mean would flatter it at 0.5.

F1 assumes precision and recall matter equally, which is rarely true. F-beta lets you weight:

from sklearn.metrics import fbeta_score

fbeta_score(y_true, y_pred, beta=2.0)    # beta > 1 favours RECALL
fbeta_score(y_true, y_pred, beta=0.5)    # beta < 1 favours PRECISION

beta is how many times more you value recall than precision. beta=2 for screening (missing a case is expensive), beta=0.5 when false alarms are expensive.

The direction of beta is a common slip: bigger beta means recall matters more.

Choosing the operating point

Optimising F1 is a default, not an answer. Better approaches, in order of how well they’ll land in an interview:

Constraint-based — the reviewing team can handle 200 cases a day:

k = 200
top_k = np.argsort(scores)[-k:]
precision_at_k = y_val[top_k].mean()      # what fraction of the queue is real

Precision floor — the business won’t tolerate below 30% precision:

from sklearn.metrics import precision_recall_curve
import numpy as np

precision, recall, thresholds = precision_recall_curve(y_val, scores)
ok = precision[:-1] >= 0.30
threshold = thresholds[ok][np.argmax(recall[:-1][ok])]   # max recall subject to that

Cost-based — the strongest answer when you can get the numbers:

# missed fraud costs 100x a false alarm
best = min(candidate_thresholds,
           key=lambda t: 100 * fn_at(t) + 1 * fp_at(t))

Note the shapes of precision_recall_curve output: precision and recall have one more element than thresholds, which is why the slicing above uses [:-1]. Getting that wrong shifts your chosen threshold by one and is easy to miss.

The precision-recall curve

Sweep every threshold and plot precision against recall. The area under it — average precision — summarises the model independently of any single operating point:

from sklearn.metrics import average_precision_score
average_precision_score(y_val, scores)

A random classifier’s PR curve sits flat at the positive base rate, so PR-AUC of 0.1 on a 10%-positive dataset is no better than chance. Unlike ROC-AUC, the PR baseline moves with class balance, so you must state the base rate when reporting it. Detail in 03_roc_auc_vs_pr_auc.md.

Which to prioritise

The two-stage pattern is worth naming: a high-recall first stage generates candidates cheaply, a high-precision second stage (a heavier model, or a human) filters them. This is exactly how retrieval-then-rerank works in search and RAG, and how fraud triage queues work. It dissolves the trade-off by applying each metric where it belongs.

Multi-class and multi-label

For multi-class, compute per class then average — see 01_confusion_matrix_accuracy.md for micro/macro/weighted.

For multi-label, each label gets its own threshold. A single global threshold across labels with very different base rates is a common and quietly costly mistake.

Interview angle

• “Precision or recall for a cancer screening test?” — recall. A missed cancer is far worse than a false alarm that leads to a follow-up test. Note that screening is deliberately the high-recall first stage of a two-stage process, with a more precise diagnostic behind it.
• “How do you improve precision?” — first ask whether they mean at the same operating point. Raising the threshold trades recall for precision immediately. Genuinely improving both needs a better model, better features, or more data — that’s what raises the whole PR curve.
• “Why harmonic mean in F1?” — it punishes imbalance. A model with precision 1.0 and recall 0.01 gets F1 ≈ 0.02, while the arithmetic mean would report a misleading 0.5.
• “When is F1 the wrong metric?” — whenever the two errors have different costs, which is most of the time. Use F-beta with a justified beta, or better, minimise expected cost directly.
• “How do you actually choose the threshold?” — from a business constraint: review capacity (precision@k), a precision floor, or a cost ratio. Not from F1, and never left at the 0.5 default.
• “PR-AUC is 0.4. Good?” — depends entirely on the base rate. At 40% positives that’s chance; at 1% positives it’s excellent. Always report PR-AUC with the positive rate beside it.