Free Practice Quiz Question List

7 Model Evaluation Online Quiz Questions

Use this free practice quiz with 20 questions to review 7 Model Evaluation, test your knowledge, and prepare for your next test or exam.

20 questions

Written response

1 point

A dataset has 100 cases, of which 1 is positive. A classifier predicts every case as negative. What is its accuracy, as a percentage?

Choose one

1 point

A team is developing a predictive model and wants to compare alternatives, tune settings, and then estimate final performance. Which plan best assigns these roles to its data?

A
Fit the model on validation data, tune it on the test set, and report training performance as the final estimate.
B
Fit the model on training data, compare models and tune settings on validation data, and use a separate test set for final evaluation.
C
Use the test set for repeated model comparisons, then use the validation set as the final unbiased estimate.
D
Combine training, validation, and test data before fitting so each split contributes equally.

Choose one

1 point

A regression task needs an intuitive summary of typical prediction error in the outcome's units. A few unusually large misses should not dominate the score. Which measure is the best fit?

A
Choose RMSE because it gives less influence to extreme errors than MAE.
B
Choose MSE because it is expressed in the same units as the outcome.
C
Choose MAE because it summarizes typical error in the outcome's units and is less affected by a few very large errors.
D
Choose R² because it is an error measure expressed in the outcome's units.

Written response

1 point

A screening system is evaluated by the fraction of all actual positive cases that it identifies. Which classification metric describes this?

Choose one

1 point

In a dataset where only 1 in 100 cases is positive, a classifier that predicts every case as negative achieves 99% accuracy. What is the most appropriate interpretation?

A
The model has strong positive-case detection because its accuracy is high.
B
The accuracy is misleading by itself: the model finds no positives, so class-specific measures should also be examined.
C
The model has perfect recall because it predicts every case as negative.
D
The model's accuracy proves that its predicted probabilities are well calibrated.

True or false

1 point

The F1 score accounts for true negatives as well as balancing precision and recall.

A
True
B
False

Choose one

1 point

A team wants to compare how well classifiers rank positive cases above negative cases across possible thresholds, without evaluating just one chosen threshold. Which measure best matches that goal?

A
ROC-AUC summarizes how well a classifier ranks positive examples above negative ones across thresholds.
B
Accuracy measures the fraction of labels that are correct at one fixed threshold.
C
Precision measures the fraction of predicted positives that are truly positive.
D
Mean absolute error measures average absolute error for numerical predictions.

Written response

1 point

A classifier has $TN=72$ and $FP=8$ . Using specificity $=\frac{TN}{TN+FP}$ , what is its specificity? Enter the result as a decimal rounded to two decimal places.

Choose one

1 point

A classifier assigns labels by comparing scores with a threshold. The team must choose a threshold that meets application needs and still obtain a final performance estimate. What is the best procedure?

A
Choose the threshold using test-set results, then report those same results as an unbiased final estimate.
B
Choose the threshold using training data only, then skip testing the final choice.
C
Select a threshold on validation data to meet the application's needs, then evaluate the final choice on the test set.
D
Use the test set repeatedly to adjust the threshold until the desired score is reached.

Choose one

1 point

A regression model has a negative $R^2$ on a test set. What does this indicate about its performance relative to the mean-value baseline?

A
The model predicts every test outcome perfectly.
B
The model performs exactly as well as predicting the test-set mean.
C
The model's average absolute error is negative.
D
The model performs worse than predicting the test-set mean.

Choose one

1 point

A forecasting team needs predicted probabilities to be meaningful—for example, cases assigned a probability of 0.8 should correspond to about an 80% event rate. What should the team do?

A
Use accuracy alone, because correct labels establish that predicted probabilities are reliable.
B
Evaluate probability quality with a measure such as log loss and check calibration.
C
Use ROC-AUC alone, because ranking quality guarantees calibrated probabilities.
D
Use MAE, because it directly measures classification probability calibration.

Choose one

1 point

A regression model predicts 11 when the actual value is 14. What is the residual?

A
-3
B
3
C
11
D
14

Choose all

1 point

A classifier has precision of 90% and recall of 40%. Select all conclusions that follow from these values.

A
Of the cases predicted positive, 90% were truly positive.
B
Of the actual positive cases, 90% were found.
C
Of the actual positive cases, 60% were missed.
D
Of the cases predicted negative, 40% were truly negative.

Choose all

1 point

A team is preparing an evaluation report for a model. Select all practices that would make the evaluation more informative and relevant to its intended use.

A
Evaluate on data representative of the intended use.
B
Compare performance with a simple baseline.
C
Report only the highest score and omit how the metric is defined.
D
Report metric definitions and relevant test-set results.

True or false

1 point

True or false: Specificity can be useful when false positives are a particular concern.

A
True
B
False

True or false

1 point

True or false: A single evaluation score is a complete description of a model's quality.

A
True
B
False

Written response

1 point

For two observations, the actual values are 8 and 13, and the predictions are 10 and 10. What is the mean absolute error?

Fill in the blank

1 point

For actual value $y_i$ and prediction $\hat{y}_i$ , the residual is $e_i =$ .

Fill in the blank

1 point

For an evaluation to be relevant to deployment, use data of the intended use and compare the model with a simple .

Open ended

1 point

A team is preparing to use a model in a new setting. Describe a sound evaluation plan that would help them judge whether its performance is useful there. Explain how the decision and possible errors should influence the evaluation, and identify what context should be included when reporting results.