Multiple regression and diagnostics - Model specification fu...

Learning Outcomes

This article explains how to specify, test, and interpret multiple regression models for CFA Level 2, including:

• recognizing forms and sources of model misspecification in cross-sectional and time-series regressions;
• assessing the impact of omitted variables, incorrect functional form, and inappropriate scaling on coefficient estimates, standard errors, and inference;
• selecting and applying suitable functional forms (linear, log-linear, polynomial) and variable transformations for financial data;
• interpreting coefficients in transformed models, including elasticity-style interpretations for log specifications;
• constructing, estimating, and interpreting interaction terms, and explaining how they modify marginal effects across groups or variable levels;
• using basic diagnostic tools—residual plots, goodness-of-fit measures, and specification tests—to evaluate model adequacy and detect non-linearity or missing interactions;
• linking each diagnostic result and specification choice to typical CFA exam questions involving yield spreads, valuation multiples, forecasting applications, and risk modeling scenarios, in both conceptual and calculation-based formats;
• integrating these skills to evaluate published regression results and to design exam-ready modeling approaches.

CFA Level 2 Syllabus

For the CFA Level 2 exam, you are required to understand the critical decisions in specifying multiple regression models, with a focus on the following syllabus points:

• explaining how incorrect model specification impacts regression results and inference;
• identifying when a functional form or transformation (e.g., log or polynomial) is appropriate for variables;
• recognizing omitted variable bias and its implications for coefficient estimates;
• applying interaction terms and explaining their interpretation in financial contexts;
• using variable scaling and functional form adjustments for correct model fit;
• comparing alternative model specifications using adjusted $R^2$, information criteria, and joint hypothesis ($F$) tests.

Test Your Knowledge

Attempt these questions before reading this article. If you find some difficult or cannot remember the answers, remember to look more closely at that area during your revision.

An analyst is modeling the cross-sectional determinants of corporate bond yield spreads (Spread) using data for 300 firms. Her initial model is:

where:

• $\text{Lev}$ = debt-to-assets ratio (in %),
• $\text{Size}$ = total assets (in millions of USD),
• $\text{IG}$ = 1 if the issuer is investment grade, 0 otherwise.

After examining scatter plots and residuals, she considers alternative specifications and additional variables.

1. Residual plots against Size show a clear curvature: spreads fall rapidly as Size increases from small to medium firms, then flatten for very large firms. Which adjustment is most appropriate to address this issue?

   a) Drop Size from the model because it is non-linearly related to Spread.
   1. Replace Size with log⁡(Size)\log(\text{Size})log(Size) in the regression.
   2. Add Size squared but keep Size measured in millions.
   3. Standardize Size by subtracting its mean and dividing by its standard deviation.

2. The analyst suspects that leverage has a different impact on Spread for investment-grade (IG) versus non-investment-grade issuers. She augments the model with an interaction term:

Which statement best describes $b_3$?

a) The effect of leverage on Spread for non-investment-grade issuers.
b) The average difference in Spread between IG and non-IG issuers, holding leverage constant.
c) The incremental effect of leverage on Spread for IG issuers relative to non-IG issuers.
d) The total effect of leverage on Spread for IG issuers.

1. To check whether omitting a liquidity variable (BidAsk) materially affects the model, she estimates:

• Restricted model: Spread on Lev, log(Size), IG
• Unrestricted model: Spread on Lev, log(Size), IG, BidAsk

The sum of squared errors (SSE) falls only slightly when BidAsk is added, and the $F$-test for the additional variable is not significant. Which conclusion is most justified?

a) BidAsk is economically important and should be kept despite statistical insignificance.
b) BidAsk can be omitted; the restricted model is more parsimonious without meaningful loss of fit.
c) The presence of BidAsk guarantees removal of all omitted variable bias.
d) The analyst must choose the unrestricted model because it has the lower SSE.

1. The analyst considers taking logs of both Spread and Size, estimating:

If $b_1 = -0.40$, how should it be interpreted?

a) A one-unit increase in Size reduces Spread by 0.40 basis points.
b) A 1% increase in Size is associated with a 0.40% decrease in Spread.
c) A 1% increase in Size is associated with a 0.40 basis point decrease in Spread.
d) A one-standard-deviation increase in Size reduces Spread by 0.40%.

Introduction

Specifying an appropriate multiple regression model is essential for valid and reliable results. Poor specification—omitting key variables, misusing the form of variables, or neglecting meaningful interactions—can result in biased estimates, misleading significance tests, and faulty predictions. This is exactly the type of issue that Level 2 exam cases test: you are shown regression output and asked to evaluate whether the model supports an investment conclusion.

Key Term: model misspecification
Model misspecification occurs when the regression model omits relevant variables, uses incorrect functional forms, or fails to capture variable interactions, leading to biased or inconsistent parameter estimates and unreliable inference.

At Level 2, the emphasis is on application and diagnostics:

• Can you recognize when a variable should enter in logs rather than levels?
• Do you understand how an interaction term changes the interpretation of a coefficient?
• Can you tell, from residual patterns or model comparisons, that the specification is inadequate?

The sections that follow develop these skills, using typical financial applications: explaining yield spreads, valuation multiples, or time-series trends such as loan growth or sales.

Model specification and functional forms

A well-specified regression model:

• includes all independent variables with a sound economic rationale;
• uses transformations that best describe the actual relationships;
• captures important non-linearities and interactions;
• is parsimonious (no redundant or irrelevant variables);
• performs well out-of-sample and respects core regression assumptions.

Key Term: functional form
Functional form is the mathematical relationship assumed between the dependent and independent variables (e.g., linear, log-linear, or polynomial).

Key Term: variable transformation
A variable transformation changes how a variable enters the model, using operations such as logs, differences, or powers, to better represent its relationship with the dependent variable or stabilize variance.

Mis-specifying the functional form can be just as damaging as omitting a variable: the coefficients may be biased, standard errors may be wrong, and residuals often show patterns (curvature, changing variance) that violate regression assumptions.

Omitted variable bias

Omitting a relevant independent variable causes biased and inconsistent coefficient estimates for the included variables, especially if the omitted variable is correlated with variables in the model.

Key Term: omitted variable bias
Omitted variable bias is the distortion in estimated coefficients that occurs when a relevant explanatory variable is left out of the regression, and the omitted variable is correlated with one or more included regressors.

Two cases are important:

• Omitted variable correlated with included regressors:
  Suppose the true model for stock returns is

  $$R_i = b_0 + b_1 \beta_i + b_2 \ln(\text{MktCap}_i) + \varepsilon_i$$

  but you estimate a model without firm size:

  $$R_i = a_0 + a_1 \beta_i + u_i$$

  If $\ln(\text{MktCap})$ is correlated with $\beta$, the error term $u_i$ absorbs the omitted size effect and becomes correlated with $\beta$. OLS then yields biased and inconsistent estimates of $a_1$. Hypothesis tests on $\beta$ (e.g., whether it is priced) will be unreliable.

• Omitted variable uncorrelated with included regressors:
  If the omitted variable is statistically independent of the included regressors, the slope coefficients remain unbiased, but the intercept is biased and the error variance increases. You lose explanatory power but avoid omitted-variable bias on the included slopes.

In practice, you do not know the “true” model, so you rely on:

• Economic theory and institutional knowledge (e.g., credit risk typically depends on leverage, profitability, size, and liquidity);
• Specification tests, especially joint $F$-tests on additional variables.

Key Term: nested models
Nested models are models where the restricted model is obtained from the full model by imposing linear restrictions on the coefficients (usually by setting some coefficients to zero).

For nested models, the joint $F$-test compares a restricted model (fewer variables) with an unrestricted model (additional variables). If the decrease in SSE from adding variables is statistically significant, the more complex specification is justified.

Inappropriate functional form

Not all relationships between variables are linear. Some may be better modeled with transformations such as logarithms or powers.

A classic example is the effect of firm size on returns or yield spreads. The first billion of market capitalization may reduce risk substantially, but the marginal effect of additional billions quickly diminishes.

If a variable’s effect on the dependent variable increases at a decreasing rate, a log or polynomial transformation may be appropriate. For example, modeling the effect of firm size often uses $\log(\text{Size})$, as the marginal effect diminishes at higher values.

Key Term: log-linear model
A log-linear model is a regression where the dependent variable is in logs and the independent variables are in levels (log–level), or vice versa (level–log), or both are in logs (log–log). The interpretation of coefficients depends on which variables are logged.

Key Term: polynomial term
A polynomial term is a regressor that is a power of an independent variable (e.g., $X^2$ or $X^3$), used to capture curvature and non-linear effects in an otherwise linear regression.

In addition to logs and polynomials, you can use piecewise linear functions (splines) or interactions to capture changes in slope beyond different thresholds.

Typical functional forms and interpretations

Consider $Y$ as the dependent variable and $X$ as a regressor:

• Level–level:

  $$Y = b_0 + b_1 X + \varepsilon$$

  Interpretation: a one-unit increase in $X$ changes $Y$ by $b_1$ units.

• Log–level:

  $$\ln Y = b_0 + b_1 X + \varepsilon$$

  Approximate interpretation: a one-unit increase in $X$ changes $Y$ by roughly $100 \times b_1$ percent.

• Level–log:

  $$Y = b_0 + b_1 \ln X + \varepsilon$$

  Interpretation: a 1% increase in $X$ changes $Y$ by approximately $0.01 b_1$ units.

• Log–log (elasticity form):

  $$\ln Y = b_0 + b_1 \ln X + \varepsilon$$

  Interpretation: $b_1$ is an elasticity; a 1% increase in $X$ is associated with a $b_1$% change in $Y$.

This is frequently used in Level 2 contexts such as relating volatility to size, or trading volume to spreads.

Key Term: heteroskedasticity
Heteroskedasticity occurs when the variance of the regression residuals is not constant across observations, often visible as a “fan shape” in residual plots.

Key Term: serial correlation
Serial correlation (autocorrelation) occurs when regression residuals from time-series data are correlated across time, violating the assumption of independent errors.

Incorrect functional form often produces systematic patterns in residual plots and may induce heteroskedasticity or serial correlation in time-series models. For example, fitting a linear trend to an exponentially growing series (e.g., loan volumes) yields residuals that are negative early and positive later, and a Durbin–Watson statistic far from 2, signaling autocorrelation.

Scaling and standardization

Incorrect variable units or lack of scaling can artificially inflate variances or introduce spurious relationships, making results unreliable. Analysts frequently use ratios (e.g., debt/EBITDA) or standardize variables (e.g., dividing by total assets) to address scaling.

Key Term: variable scaling
Variable scaling refers to rescaling raw variables—using ratios, percentages, or standardized values—to make magnitudes comparable across observations and avoid numerical instability or misleading relationships.

Common reasons to rescale include:

• Comparability across firms:
  Using leverage as debt/total assets rather than absolute debt allows comparison of firms of different sizes.
• Reducing heteroskedasticity:
  Earnings in dollars often have variance that increases with firm size; using ROE or profit margin instead stabilizes variance.
• Numerical stability:
  Very large values (e.g., market cap in millions) can cause software rounding issues; expressing in billions or logs helps.

Key Term: dummy variable
A dummy (indicator) variable takes the value 1 if a categorical condition holds and 0 otherwise, allowing qualitative characteristics (such as industry or credit rating) to be included in regression models.

Scaling does not change statistical significance in a fundamental sense, but it changes the magnitude and units of coefficients. For example, using leverage in % rather than proportion multiplies the coefficient by 100.

Interaction terms

In some situations, the effect of one independent variable may depend on the value of another. Interaction terms allow the regression to model these cases.

Key Term: interaction term
An interaction term is the product of two independent variables included in a regression to test whether the effect of one variable depends on the level or category of another variable.

In financial analysis, interaction terms help answer whether the effect of a variable changes:

• across groups (e.g., IG vs non-IG issuers),
• across regimes (e.g., crisis vs normal times),
• with the level of another continuous variable (e.g., size modifies the impact of leverage).

Two common types:

• Dummy × continuous interaction:
  Allows the slope on a continuous variable to differ between groups.
  Example:

  $$\text{Spread} = b_0 + b_1 \text{Lev} + b_2 \text{IG} + b_3 (\text{Lev} \times \text{IG}) + \varepsilon$$
  • For non-IG ($\text{IG}=0$): slope on Lev is $b_1$
  • For IG ($\text{IG}=1$): slope on Lev is $b_1 + b_3$

• Continuous × continuous interaction:
  Allows the marginal effect of $X_1$ to vary with $X_2$:

  $$Y = b_0 + b_1 X_1 + b_2 X_2 + b_3 (X_1 X_2) + \varepsilon$$

  The partial effect of $X_1$ is $b_1 + b_3 X_2$; it is state-dependent.

These interactions are common in Level 2 questions involving, for example, whether leverage becomes more “dangerous” in recessions, or whether a governance score matters more for small than for large firms.

Worked Example 1.1

A CFA candidate is analyzing the determinants of bond yield spreads. She believes that the size premium depends not only on firm debt ratio but also on whether the issuer is investment grade. She specifies:

Question: What does the coefficient $b_3$ measure?

Answer:
$b_3$ measures the additional effect of the debt ratio on yield spreads for investment-grade issuers. In other words, it is the difference in the leverage slope between investment-grade and non-investment-grade bonds. If $b_3 < 0$, leverage increases spreads less for IG bonds than for speculative-grade bonds; if $b_3 > 0$, leverage is penalized more heavily for IG issuers.

Worked Example 1.2

Suppose you wish to model the effect of advertising expenditure (ADV) on sales (SALES), but preliminary plots suggest diminishing marginal returns. Which model is most appropriate:

• (a) $\text{SALES} = b_0 + b_1 \text{ADV} + \varepsilon$, or
• (b) $\text{SALES} = b_0 + b_1 \log(\text{ADV}) + \varepsilon$?

How can you confirm your choice?

Answer:
Model (b), $\text{SALES} = b_0 + b_1 \log(\text{ADV}) + \varepsilon$, is more appropriate. Applying $\log(\text{ADV})$ allows the marginal effect of advertising to diminish as ADV rises, which matches the observed curvature. You can confirm this by:

• comparing residual plots (the correct form should yield residuals with no obvious curvature);
• comparing adjusted $R^2$ and information criteria (AIC/BIC) across the two models;
• testing whether additional non-linear terms (e.g., ADV-squared in model (a)) add significant explanatory power.

Diagnostics and model comparison

To decide whether a functional form or interaction is empirically supported, you rely on diagnostics and model comparison tools.

Key Term: adjusted R-squared
Adjusted $R^2$ is a goodness-of-fit measure that penalizes adding regressors; it increases only if a new variable improves the model more than expected by chance.

The formula is:

where $n$ is the number of observations and $k$ the number of independent variables. When you add a variable:

• $R^2$ always weakly increases;
• $R^2_a$ may increase or decrease, depending on whether the new variable is useful.

Key Term: Akaike information criterion (AIC)
AIC is an information criterion that balances model fit (via SSE) and complexity (number of regressors); lower AIC indicates a preferable model for forecasting.

Key Term: Bayesian information criterion (BIC)
BIC is a stricter information criterion than AIC, imposing a heavier penalty on additional regressors; lower BIC suggests a better-fitting model in-sample.

Typical formulas are:

Both criteria use SSE (lower is better) and penalize overfitting via $k$ (higher is worse). BIC’s penalty grows with sample size, so it is particularly conservative about adding regressors in large samples.

For nested models, you can also use a joint $F$-test on the additional terms (e.g., interaction terms or polynomial terms) to decide whether the more complex specification is warranted.

Exam application: A Level 2 item set might show three competing models for rental prices or P/E ratios, providing $R^2_a$, AIC, and BIC. If the vignette asks which model is best for forecasting, choose the one with the lowest AIC; if it asks which has best in-sample fit, consider BIC and adjusted $R^2$.

Time-series functional form: linear vs log-linear trends

Although this reading is about multiple regression, Level 2 also expects you to apply the same functional form logic to time-series trends:

• If a series increases by a constant absolute amount (e.g., loans grow by about $10 billion per quarter), a linear trend model

  $$x_t = b_0 + b_1 t + \varepsilon_t$$

  is appropriate.

• If a series grows at a constant percentage rate (e.g., sales grow roughly 5% per year), a log-linear specification

  $$\ln x_t = b_0 + b_1 t + \varepsilon_t$$

  is better; the slope $b_1$ approximates the growth rate.

Choosing the wrong functional form can produce autocorrelated residuals (low Durbin–Watson statistic) and unreliable forecasts.

Worked Example 1.3

An analyst regresses the natural log of a stock’s bid–ask spread on the natural log of its average daily trading volume:

The estimate is $b_1 = -0.25$, significant at the 1% level.

Question: Interpret $b_1$ and explain why this log–log form might be preferred over a level–level regression.

Answer:
In a log–log model, $b_1$ is an elasticity. Here, $b_1 = -0.25$ means that a 1% increase in trading volume is associated with a 0.25% decrease in bid–ask spreads, on average. This form is often preferred because:

• economic theory suggests a proportional (percentage) relationship between liquidity and transaction costs;
• logging both variables reduces skewness and can stabilize variance (mitigating heteroskedasticity);
• residual plots for the log–log model typically show less curvature than for a level–level model.

Worked Example 1.4

An analyst models return on equity (ROE) using leverage (Lev) and a crisis dummy (Crisis = 1 during crisis years, 0 otherwise):

Estimated coefficients are $b_1 = 0.10$, $b_2 = -0.03$, and $b_3 = -0.08$ (all statistically significant).

Question: How does leverage affect ROE in normal vs crisis periods?

Answer:
For non-crisis periods ($\text{Crisis}=0$), the slope on leverage is $0.10$: a 1-unit increase in Lev (e.g., from 1.0 to 2.0 debt-to-equity) is associated with a 0.10 increase in ROE.
For crisis periods ($\text{Crisis}=1$), the slope becomes $0.10 + (-0.08) = 0.02$. Thus, the positive impact of leverage on ROE is much weaker in crises. The interaction coefficient $b_3 = -0.08$ measures this reduction in the leverage slope during crises.

Worked Example 1.5

You are modeling rental price per square foot (RENT) as a function of building age (AGE). A linear model yields:

Residual plots show that for very new and very old buildings, the model systematically underpredicts rents, while for mid-age buildings it overpredicts rents, suggesting curvature. You consider adding a squared term:

Question: How would you test formally whether the polynomial specification is justified?

Answer:
Estimate both models on the same sample:

• Restricted (linear): RENT on AGE
• Unrestricted (quadratic): RENT on AGE and AGE²
  Then perform a joint $F$-test with null hypothesis $H_0: b_2 = 0$. The test statistic compares SSE from the two models: $$F = \frac{(\text{SSE}_R - \text{SSE}_U)/1}{\text{SSE}_U/(n - k_U - 1)}$$ where $\text{SSE}_R$ and $\text{SSE}_U$ are the restricted and unrestricted SSEs, $k_U$ is the number of regressors in the unrestricted model. If $F$ exceeds the critical value, you reject $H_0$ and conclude that the AGE² term adds statistically significant explanatory power, supporting the polynomial specification.

Exam warning: A frequent error is assuming linearity for all variables. Using an incorrect form—such as a linear specification for exponentially growing revenue—can produce biased coefficients and spurious significance. Before modeling, examine scatter plots and residuals, and consider transformations—especially for variables measured in large monetary units or that may enter non-linearly.

Summary

Careful model specification is essential for sound regression analysis. Omitting key variables, neglecting transformations, or failing to include relevant interactions all introduce errors into financial models. Always assess the economic rationale for included variables and test functional form validity. Using suitable transformations and interaction terms helps ensure your regression model produces credible, interpretable results for the CFA exam and in professional finance practice.

Key Point Checklist

This article has covered the following key knowledge points:

• Principles of model specification: economic rationale, parsimony, and respect for regression assumptions.
• Omitted variable bias: when it arises, how it affects coefficients and inference, and how joint $F$-tests on added variables help detect it.
• Functional forms: when to use linear, log-linear, and polynomial specifications; how to interpret coefficients in each.
• Diagnostics for functional form: use of residual plots, adjusted $R^2$, AIC, BIC, and joint tests for additional terms (e.g., polynomials, interactions).
• Scaling and standardization: using ratios, percentages, and logs to improve comparability and reduce heteroskedasticity.
• Interaction terms: construction and interpretation for dummy × continuous and continuous × continuous interactions; impact on marginal effects.
• Time-series functional form: choosing between linear and log-linear trend models based on constant level vs constant growth patterns.
• Exam-focused application: linking specification choices to typical CFA Level 2 vignettes involving yield spreads, valuation multiples, loan growth, and profitability models.

Key Terms and Concepts

• model misspecification
• functional form
• variable transformation
• omitted variable bias
• nested models
• log-linear model
• polynomial term
• heteroskedasticity
• serial correlation
• variable scaling
• dummy variable
• interaction term
• adjusted R-squared
• Akaike information criterion (AIC)
• Bayesian information criterion (BIC)