8 Generalization and Overfitting Free Online FlashCards

What does it mean for a model to generalize?

A model generalizes when it makes useful predictions on examples it did not train on.

What is overfitting?

Overfitting occurs when a model fits training data closely, including noise or peculiarities, but performs substantially worse on new data.

What is underfitting?

Underfitting occurs when a model is too limited or insufficiently trained to capture important patterns, so it performs poorly on both training and new data.

What score pattern suggests a useful fit?

Good training and validation performance with a modest gap suggests a useful fit that may generalize.

What can help an underfitting model?

Try a more expressive model, useful features, or a better training procedure, and check that training is adequate.

How can regularization help with overfitting?

Reduce model complexity, remove unhelpful features, or use regularization, which penalizes complexity. L2 regularization discourages excessively large weights.

What roles should validation data and a test set serve?

Use validation data or cross-validation to compare models and settings; reserve a separate test set for final evaluation.

How can you prevent preprocessing data leakage?

Split data before learning preprocessing steps. Fit transformations such as scaling or imputation on training data only, then apply them to validation and test data.

Why is training performance alone insufficient to assess generalization?

Training performance alone cannot show whether a model learned broadly applicable patterns or memorized details of its training data.

When can a test set estimate performance on future data?

A held-out test set estimates future performance when its examples are relevant to intended use and the data distributions are sufficiently similar.

What score pattern often signals overfitting?

Strong training performance paired with much weaker validation performance is a common sign of overfitting.

How can a high-degree polynomial overfit a small, noisy dataset?

A high-degree polynomial may pass through every point in a small, noisy training set yet bend sharply between points and predict poorly on new observations.