8 Introduction to Machine Learning

How machine learning uses data

Machine learning uses data to build models that identify patterns and produce predictions or descriptions. A dataset is often organized into examples, represented by rows, and features, represented by input columns. In supervised learning, each example also has a label: the value the model is intended to predict. A model is fitted using examples and then used to make predictions about new ones.

Supervised and unsupervised learning

Supervised learning learns from examples with known labels and aims to predict labels for new examples. Its two common tasks are classification, which predicts a category, and regression, which predicts a numerical value. For example, classification can predict whether an email is spam, while regression can predict a home's sale price.

Unsupervised learning works with data that has no target labels and looks for structure in the features themselves. A clustering algorithm might group customers with similar purchasing patterns. Such groups are patterns found by the algorithm, not automatically meaningful categories, so they require interpretation.

Training and evaluating on separate data

During training, an algorithm uses training data to fit a model, for example by adjusting model parameters to reduce prediction errors. A test set contains separate examples and is used to estimate how well the fitted model performs on data it has not seen. A validation set may also be used during development to compare models or tune choices.

Keep the test set separate until final evaluation. Repeatedly using it to guide decisions can make it less independent as a check. The split should reflect how the model will be used: to predict future demand, for example, testing on later dates may be more appropriate than randomly mixing past and future records.

Check for duplicates and data leakage: training features should not contain information that would not actually be available when making a prediction. If a test set is unrepresentative of real-world cases, it can give a misleading estimate.

Generalization and model limitations

A model generalizes when it performs well on new examples. If it performs poorly even on training data, it may be underfitting. If it fits training data very closely but performs poorly on unseen data, it may be overfitting. Overly complex models and unrepresentative data can contribute to overfitting.

A good test score is evidence about performance on data like the test set, not a guarantee of success in every setting. Results can be limited by small or biased samples, noisy or incorrect labels, missing information, and changes in data over time. Even an accurate model may fail when real-world data differs from the training data.

Choosing evaluation measures

Choose evaluation measures that match the prediction task and the consequences of different errors. No single score gives a complete picture.

For classification, a confusion matrix counts true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). Common measures are:

• Accuracy is the fraction of predictions that are correct:

  $$\text{Accuracy} = \frac{\mathrm{TP} + \mathrm{TN}}{\mathrm{TP} + \mathrm{TN} + \mathrm{FP} + \mathrm{FN}}$$

  Accuracy can look high when one class is much more common than another.

• Precision is the fraction of predicted positives that are truly positive:

  $$\text{Precision} = \frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FP}}$$

• Recall is the fraction of actual positives that the model detects:

  $$\text{Recall} = \frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FN}}$$

• F1 score combines precision and recall and is high when both are high:

  $$\text{F1} = \frac{2 \times \text{precision} \times \text{recall}}{\text{precision} + \text{recall}}$$

The measure to emphasize depends on the task. In a screening task where missing a true case is costly, recall may deserve particular attention; when false alarms are costly, precision may matter more. Accuracy alone may conceal poor performance on a rare class.

For regression, mean absolute error (MAE) averages absolute errors, while mean squared error (MSE) averages squared errors and therefore penalizes large errors more heavily. Root mean squared error (RMSE) is the square root of MSE and is expressed in the target's units. Lower MAE, MSE, or RMSE indicates smaller errors on the evaluated data.

Unsupervised results are harder to evaluate because there may be no known labels for comparison. A clustering result can be inspected for useful, stable patterns or evaluated against known labels when such labels exist. A numerical score should not replace checking whether the groups are meaningful for their intended purpose.

Interpreting predictions in context

Simple models are useful starting points, but their assumptions and capacity matter. A model that is too simple may miss important patterns, while a more complex model may fit noise and overfit. Evaluation measures summarize selected aspects of performance, not every practical cost or consequence.

Interpret predictions in context and inspect errors. Do not treat a model's output as proof of cause or certainty.