3 Classification Fundamentals

Classification and its classes

Classification is a type of supervised machine learning: a model learns from examples with known categories and then predicts categories for new inputs. The categories are called classes. A task with two possible classes is binary classification, while a task with more than two classes is multiclass classification.

Examples, labels, and features

A labeled example pairs an input with its known answer, or label. Features are the measurable properties used to describe the input. For an email spam classifier, features might include whether the message contains a link, the number of words in its subject, and how often certain terms appear; the label is spam or not spam.

During training, a learning algorithm uses many labeled examples to fit a model. The model can then use features from an unseen email to predict its label. The label is the target to predict, not a feature to supply to the model. Training data should represent the task well, and the features used should be available when predictions are made.

Evaluating predictions

To estimate how well a classifier works on new examples, keep some labeled data separate from training as a test set. Compare each prediction on this held-out data with its known label.

A confusion matrix summarizes binary classification results by comparing predicted and actual classes:

• True positive (TP): predicted positive and actually positive.

• False positive (FP): predicted positive but actually negative.

• False negative (FN): predicted negative but actually positive.

• True negative (TN): predicted negative and actually negative.

For example, consider a test set of $100$ emails. Of $20$ actual spam messages, the classifier correctly identifies $12$ and misses $8$. Of $80$ non-spam messages, it incorrectly flags $3$ as spam and correctly leaves $77$ unflagged. The results are $\mathrm{TP}=12$, $\mathrm{FN}=8$, $\mathrm{FP}=3$, and $\mathrm{TN}=77$.

Accuracy, precision, and recall

Accuracy measures the fraction of all predictions that are correct:

For the email example, accuracy is $\frac{12+77}{100}=89\%$.

Precision measures the fraction of positive predictions that are correct:

In the example, precision is $\frac{12}{15}=80\%$. Recall measures the fraction of actual positive examples that are found:

In the example, recall is $\frac{12}{20}=60\%$.

Accuracy alone can be misleading when one class is much more common than another. For instance, a model that predicts “not spam” for every email in a dataset that is mostly non-spam may have high accuracy but fail to identify spam. Precision matters when false alarms are costly, while recall matters when missing positive cases is costly. Choose measures that fit the task, and inspect errors by class rather than relying on one score.