Learning paths

Learning paths

Views

Full curriculumExplore every unit

Learning path

Full curriculum

Full curriculum
  1. Supervised learning and loss functions
  2. Training, validation, and test sets
  3. Confusion matrices, precision, recall, and F1
  4. Logistic regression for binary classification
  5. Softmax and categorical probabilities
  6. Multiclass classification strategies
  7. ROC and precision-recall curves
  8. Probability calibration and proper scoring rules
  9. Distribution shift and out-of-distribution generalization
  10. Anomaly detection
  11. Dataset bias and subgroup evaluation in machine learning
  12. An end-to-end supervised machine-learning workflow

Unit content

Confusion matrices, precision, recall, and F1

For binary classification, accuracy alone can hide which kinds of mistakes a model makes. A confusion matrix separates predictions into four counts:

  • true positives $TP$;
  • false positives $FP$;
  • true negatives $TN$;
  • false negatives $FN$.

From these counts,

$$\text{precision}=\frac{TP}{TP+FP}$$

measures how often a positive prediction is correct, while

$$\text{recall}=\frac{TP}{TP+FN}$$

measures how many actual positives are found.

The F1 score is their harmonic mean:

$$F_1=2\frac{\text{precision}\cdot\text{recall}}{\text{precision}+\text{recall}}.$$

Suppose a screening model finds 80 of 100 diseased patients and incorrectly flags 20 healthy patients. Then recall is $80/100=0.8$ and precision is $80/(80+20)=0.8$.

Which quantity matters depends on the decision. Missing a dangerous disease can make recall critical; sending every harmless email to a spam folder can make precision more important. Metrics therefore encode priorities, not just mathematical bookkeeping.