7 Regression for Business Analysis

What regression can do

Regression estimates how a business outcome varies with one or more factors, called predictors. It can help explain patterns and make predictions, such as estimating sales from advertising spend, price, and season. Regression alone does not prove that a predictor causes an outcome.

The model and its estimates

For one predictor, a Linear regression model can be written as:

Here, $y_i$ is the outcome, $x_i$ is a predictor, $\beta_0$ is the intercept, $\beta_1$ is the slope, and $\varepsilon_i$ represents variation not captured by the model. With several predictors, the model includes a term for each predictor:

A fitted model uses data to estimate its coefficients and produces predicted values, written $\hat y_i$. A Residual is the difference between an observed and predicted outcome, $y_i - \hat y_i$. Ordinary least squares chooses coefficients that minimize the sum of squared residuals.

Interpreting coefficients

Consider a retailer modeling weekly sales, in thousands of dollars, using advertising spend, in thousands of dollars, and price, in dollars:

Holding price constant, an additional $1,000 in advertising is associated with an estimated $2,400 increase in weekly sales. Holding advertising constant, a $1 increase in price is associated with an estimated $1,100 decrease in weekly sales. These are average relationships in the data, not guaranteed effects for every week or proof of causation.

Comparing predictors

In Multiple regression, each coefficient describes the association with its predictor while the other included predictors are held constant. For a categorical predictor, software typically compares each category with a chosen reference category. Check variable units, coding, and the reference group before interpreting coefficients.

Using results for decisions

Translate regression results into the business question. Consider the direction and size of each estimated relationship, and whether its size is meaningful in business units.

A Confidence interval shows a range of plausible values for a coefficient. A Prediction interval describes uncertainty for an individual future outcome and is usually wider.

A small p-value is evidence against a zero coefficient under the model. It does not show that the effect is large, useful, or causal. Consider the estimated size, uncertainty, costs, and business consequences.

Association is not causation. Customer demand, seasonality, competitor actions, or other omitted factors may affect both a predictor and the outcome. Causal claims usually require a suitable experimental or causal-inference design, not just a regression table.

Assumptions and diagnostics

Linear regression is most useful when its structure is a reasonable representation of the data. Check assumptions and data quality rather than relying only on a model summary.

1. Linearity in the modeled terms: The expected outcome should follow the specified relationship. Plot residuals against fitted values and important predictors; curves or systematic patterns suggest the model may be missing structure. “Linear” refers to the coefficients, so a model can include terms such as a squared predictor and still be linear in its coefficients.

2. Independent errors: Residuals should not show dependence, such as a pattern across successive days or repeated observations from the same customer. Business time-series and clustered data often need methods that account for this dependence.

3. Constant error variance: Residual spread should be roughly similar across fitted values. A fan-shaped pattern suggests changing variance, which can make standard errors and intervals unreliable if ignored.

4. Residual normality for classical inference: Approximate normality is useful for some small-sample confidence intervals and tests. It is not required to calculate least-squares coefficients, and it is distinct from the need for good predictive accuracy.

5. No severe Multicollinearity: Predictors that move closely together can make individual coefficients unstable and hard to interpret, even when overall predictions are adequate.

Also inspect unusual observations and influential points, confirm that data were collected and recorded appropriately, and consider whether important predictors were omitted. Residual plots are a practical first check; a high overall fit statistic cannot reveal every modeling problem.

Evaluating predictions

Evaluate performance on data that was not used to fit the model. A random train/test split can work for independent observations. For forecasting, use a chronological split so future information does not leak into the training data. Cross-validation can help compare candidate models when appropriate.

Common measures describe different aspects of prediction performance:

• MAE (mean absolute error): Average absolute prediction error, in the outcome’s units; it is easy to explain to stakeholders.

• RMSE (root mean squared error): Also in the outcome’s units, but it penalizes large errors more heavily.

• R-squared: The fraction of outcome variation explained by the fitted model in the evaluation data. It is not the percentage of predictions that are correct, and a higher value alone does not establish that a model is useful.

• Adjusted R-squared: Modifies R-squared to account for the number of predictors. It is mainly for comparing models fitted to the same response and dataset, and is not a substitute for out-of-sample validation.

Compare candidate models with a metric aligned with the cost of errors. For example, a business especially concerned about large inventory shortages may prioritize a measure that penalizes large misses. Check performance across relevant groups and time periods, and verify that predictions are plausible for their intended use. Avoid extrapolating far beyond the data used to fit the model.

Putting the pieces together

Regression estimates relationships between an outcome and its predictors and can support explanation or prediction. Interpret coefficients in context, distinguish association from causation, check assumptions with residual diagnostics, and evaluate predictions on held-out data using business-relevant metrics. A model is useful when its assumptions, accuracy, uncertainty, and intended decision all make sense together.