## **Learning Outcomes** This article explains how ordinary least squares (OLS) regression is used to estimate and test linear relationships in risk management applications. After reading, you will be able to: - Estimate and interpret simple OLS regression coefficients using sample moments - Extend OLS to multiple explanatory variables and interpret partial slope coefficients - Apply $t$-tests, confidence intervals, $R^2$/adjusted $R^2$, and $F$-tests to assess statistical and economic relevance ## **GARP FRM Part 1 Syllabus** For the GARP FRM Part 1 exam, you are expected to understand regression analysis with a focus on the following syllabus points: - Interpretation and estimation of OLS in simple linear regression, including key assumptions and estimator properties - Multiple regression structure, partial effects, goodness-of-fit measures ($R^2$ and adjusted $R^2$), and coefficient tests - Joint hypothesis testing using the $F$-test and common threats to inference (for example, heteroskedasticity and multicollinearity) ## **Test Your Knowledge** _Attempt these questions before reading this article. If you find some difficult or cannot remember the answers, remember to look more closely at that area during your revision._ 1. In simple linear regression, the OLS slope estimator can be written as: a) $b=\dfrac{\operatorname{Var}(X)}{\operatorname{Cov}(X,Y)}$ b) $b=\dfrac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(X)}$ c) $b=\dfrac{\operatorname{Cov}(X,\varepsilon)}{\operatorname{Var}(X)}$ d) $b=\dfrac{\operatorname{Var}(Y)}{\operatorname{Cov}(X,Y)}$ 2. A regression coefficient estimate is $b=1.08$ with standard error $s_b=0.67$. Using a 5% two-sided critical value of 2.02, compute a 95% confidence interval for the coefficient. 3. In a multiple regression with $n=80$ observations and $k=4$ independent variables, what are the degrees of freedom for a $t$-test of an individual slope coefficient? 4. True/False (briefly explain): If $R^2$ increases after adding a new independent variable, the new variable must be statistically significant. ## **What Do You Know Already?** - Can you explain the difference between a slope coefficient in simple regression and a partial slope coefficient in multiple regression? - Can you compute $b$ and $a$ from $\bar{X}$, $\bar{Y}$, $\operatorname{Var}(X)$, and $\operatorname{Cov}(X,Y)$ without software? - Can you describe when to use a $t$-test versus an $F$-test in regression output? Use these prompts to gauge your starting point; then focus your revision where your answers were least confident. ## **Introduction** Regression analysis is the standard tool used in finance and risk management to quantify relationships between variables. FRM-relevant examples include estimating an equity portfolio’s market beta (factor exposure), estimating a minimum-variance hedge ratio (slope of spot changes on futures changes), and evaluating manager performance (intercept as “alpha” after controlling for risk factors). For the FRM Part 1 exam, you need to be fluent in (1) what OLS is optimizing, (2) how to interpret coefficients and goodness-of-fit statistics, and (3) how to conduct and interpret hypothesis tests on coefficients—both individually ($t$-tests) and jointly ($F$-tests). You also need to know the main assumptions that justify inference and what happens when they fail (for example, heteroskedasticity affecting standard errors). > **Key Term: Linear regression** > A model that explains a dependent variable Y using one or more explanatory variables X through a linear function of unknown parameters, plus an error term: > > $$ > Y=\alpha+\beta X+\varepsilon \quad \text{(simple)} > $$ > > $$ > Y=\alpha+\sum_{j=1}^k\beta_j X_j+\varepsilon \quad \text{(multiple)} > $$ ## **Simple linear regression and OLS estimation** In simple (bivariate) regression, the population model is: $$Y_i=\alpha+\beta X_i+\varepsilon_i,\quad i=1,\ldots,n$$ where: - $Y$ is the dependent (explained) variable - $X$ is the independent (explanatory) variable - $\alpha$ is the intercept (expected $Y$ when $X=0$) - $\beta$ is the slope (sensitivity of $Y$ to $X$) - $\varepsilon$ is the unobserved error term (shock, noise, disturbance) ### Model meaning in risk management - **Factor exposure**: Regress a portfolio’s excess returns on market excess returns; $\beta$ is market exposure. - **Hedging**: Regress spot price changes on futures price changes; slope estimates the hedge ratio. - **Performance attribution**: Regress fund returns on common factors; intercept is abnormal performance after controlling for factors. A key exam point: regression treats variables asymmetrically. We model how variation in $X$ is associated with variation in $Y$, and inference is built around assumptions about $\varepsilon$. > **Key Term: Ordinary least squares (OLS)** > The estimation method that chooses coefficients to minimize the sum of squared residuals (squared prediction errors) in the sample: > > $$ > RSS=\sum_{i=1}^n (Y_i-\hat{Y}_i)^2 > $$ ### OLS fitted values and residuals Given estimated coefficients $a$ and $b$, the fitted value and residual for observation $i$ are: $$\hat{Y}_i=a+bX_i$$ $$e_i=Y_i-\hat{Y}_i$$ > **Key Term: Residual** > The sample counterpart of the unobserved error term, equal to the difference between the observed value and the fitted value: > > $$ > e_i=Y_i-\hat{Y}_i > $$ ### Closed-form OLS estimators OLS produces “moment-like” estimators that depend on sample means, variances, and covariances: $$b=\frac{\sum_{i=1}^n (X_i-\bar{X})(Y_i-\bar{Y})}{\sum_{i=1}^n (X_i-\bar{X})^2}=\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(X)}$$ $$a=\bar{Y}-b\bar{X}$$ Interpretation: - **Slope $b$**: estimated change in $Y$ for a 1-unit increase in $X$ - **Intercept $a$**: fitted value of $Y$ when $X=0$ (sometimes not economically meaningful if $X=0$ is not plausible) ### Worked Example 1.1 A stock’s average annual return is $\bar{Y}=11\%$ and the market’s is $\bar{X}=8.4\%$. The covariance and variance are: - $\operatorname{Cov}(X,Y)=132.11$ - $\operatorname{Var}(X)=181.40$ Compute $b$ and $a$, and interpret them. > **Answer:** First compute the slope: > $$b=\frac{132.11}{181.40}=0.73$$ > Then compute the intercept: > $$a=0.11-0.73\times 0.084=0.049$$ > Interpretation: A 1% increase in market return is associated with a 0.73% increase in the stock’s return (beta-like sensitivity). If the market return were 0%, the fitted stock return would be 4.9% (intercept). ## **OLS assumptions, sampling distributions, and standard errors** OLS always produces estimates, but inference (tests and confidence intervals) requires assumptions about how the data were generated. In FRM terms, these assumptions justify treating OLS estimators as unbiased/consistent with usable standard errors. ### Core assumptions for simple OLS inference A practical set of assumptions (expressed in FRM-style language) includes: 1. **Linearity**: the relationship between $Y$ and $X$ is linear in parameters. 2. **Exogeneity**: the conditional mean of the error is zero: $$E[\varepsilon_i\mid X_i]=0$$ 3. **i.i.d. sampling**: observations are independent and identically distributed (common in cross-sectional settings). 4. **Positive variance of $X$**: $\operatorname{Var}(X)>0$ (otherwise slope estimation is impossible). 5. **Homoskedasticity (constant error variance)**: $$\operatorname{Var}(\varepsilon_i\mid X_i)=\sigma^2$$ 6. **No extreme outliers**: OLS is sensitive to large outliers; extreme points can dominate the fitted line and distort inference. > **Key Term: Homoskedasticity** > The condition that the variance of the regression error is constant across observations (equivalently, residual dispersion does not systematically change with the level of X). ### What these assumptions buy you Under these conditions, the OLS estimators have standard properties emphasized in FRM preparation: - **Unbiasedness**: $E[a]=\alpha$ and $E[b]=\beta$ - **Consistency**: as $n$ grows, $a$ and $b$ converge to the true parameters - **Approximate normality**: by large-sample arguments (CLT), the sampling distributions of $a$ and $b$ are approximately normal, enabling $t$-based inference A core intuition you should remember: the slope estimator is more precise when (i) the error variance is smaller and (ii) $X$ varies more in the sample (more information about how $Y$ changes across different $X$ values). ### Standard error of the regression and coefficient standard errors Define the residual sum of squares: $$RSS=\sum_{i=1}^n e_i^2$$ The **standard error of the regression** (a sample estimate of the typical residual size) in simple regression is: $$SER=\sqrt{\frac{RSS}{n-2}}$$ The standard error of the slope in simple regression can be expressed as: $$s_b=\frac{SER}{\sqrt{\sum_{i=1}^n (X_i-\bar{X})^2}}$$ In multiple regression, the degrees-of-freedom adjustment changes to $n-k-1$ (covered later). ### Brief note on heteroskedasticity for FRM use If error variance changes systematically with $X$ (conditional heteroskedasticity), coefficient estimates remain unbiased/consistent, but standard errors from basic OLS formulas become unreliable. In that case, hypothesis tests should use heteroskedasticity-consistent (White) standard errors. ## **Hypothesis testing and confidence intervals for OLS coefficients** FRM exam questions often present coefficient estimates, standard errors, and either (i) a critical $t$ value or (ii) a p-value, then ask whether a coefficient is statistically different from a hypothesized value. ### The t-statistic for a single coefficient To test $H_0:\beta=B$ versus $H_A:\beta\neq B$ (two-sided), compute: $$t=\frac{b-B}{s_b}$$ Degrees of freedom: - Simple regression: $n-2$ - Multiple regression with $k$ regressors: $n-k-1$ A coefficient is statistically significant at level $\alpha$ if $|t|>t_{\alpha/2,\ \text{df}}$ (two-sided) or if the p-value is less than $\alpha$. ### Confidence interval for a coefficient A $(1-\alpha)$ confidence interval for $\beta$ is: $$b\pm t_{\alpha/2,\ \text{df}}\times s_b$$ Connection to hypothesis tests (exam-relevant): in a two-sided test at significance $\alpha$, you reject $H_0:\beta=B$ if and only if $B$ lies outside the $(1-\alpha)$ confidence interval. ### Worked Example 3.1 An analyst runs three separate simple regressions and obtains (estimate, standard error) pairs: - Inflation: $b=1.08$, $s_b=0.67$ - Unemployment rate: $b=-0.48$, $s_b=0.12$ Assume a 5% two-sided critical value of 2.02. 1. Compute a 95% confidence interval for the inflation slope. 2. Test $H_0:\beta=0.20$ versus $H_A:\beta\neq 0.20$ for the unemployment slope. > **Answer:** (1) Confidence interval for inflation: > $$1.08\pm 2.02\times 0.67=1.08\pm 1.3534$$ > Lower bound = 1.08-1.3534 = -0.2734; upper bound = 1.08+1.3534 = 2.4334. > So the 95% CI is approximately [-0.27, 2.43], which includes 0 (not statistically different from 0 at 5%). > > (2) Test against 0.20 for unemployment: > $$t=\frac{-0.48-0.20}{0.12}=\frac{-0.68}{0.12}=-5.67$$ > Since |-5.67| > 2.02, reject H₀. The slope is statistically different from 0.20 at the 5% level. ## **Multiple regression and partial slope interpretation** Most risk models use more than one explanatory variable. The multiple regression model is: $$Y_i=\alpha+\beta_1X_{1i}+\beta_2X_{2i}+\cdots+\beta_kX_{ki}+\varepsilon_i$$ ### Why coefficients change when variables are added In a simple regression of $Y$ on $X_1$, the slope reflects both: - the direct association of $X_1$ with $Y$, and - any association that runs through omitted variables correlated with $X_1$ When you add $X_2$, the coefficient on $X_1$ typically changes because the model now separates variation in $Y$ associated with $X_1$ from variation associated with $X_2$. > **Key Term: Partial slope coefficient** > In multiple regression, β_j measures the expected change in Y for a one-unit increase in X_j, holding all other explanatory variables constant. Example interpretation shift: - Simple model: $Y=2.0+4.5X_1$ means a 1-unit increase in $X_1$ is associated with a 4.5-unit increase in $Y$. - Multiple model: $Y=1.0+2.5X_1+6.0X_2$ means a 1-unit increase in $X_1$ is associated with a 2.5-unit increase in $Y$, **holding $X_2$ constant**. ### Additional assumption in multiple regression Multiple regression requires that explanatory variables are not perfectly linearly dependent. > **Key Term: Perfect collinearity** > A condition where one explanatory variable is an exact linear combination of others (for example, X₃=2X₁+3X₂), making OLS estimation impossible. ### Intuition for “controlling for other variables” (residualization idea) To estimate $\beta_1$ in a two-variable regression, one way to view OLS is: 1. Regress $X_1$ on $X_2$ and keep the residuals $\phi_i$ (the part of $X_1$ not explained by $X_2$). 2. Regress $Y$ on $X_2$ and keep the residuals $\lambda_i$ (the part of $Y$ not explained by $X_2$). 3. Regress $\lambda_i$ on $\phi_i$; the slope is the estimate of $\beta_1$. This construction explains why $\beta_1$ is a “net-of-$X_2$” effect. ### Dummy variables in multiple regression Dummy variables represent on/off categories and are coded as 0 or 1. In a regression with an intercept, you typically include one fewer dummy than the number of categories to avoid perfect collinearity. > **Key Term: Dummy variable** > A binary explanatory variable (0/1) used to model categorical effects (for example, industry membership, crisis vs non-crisis periods, January effect). ### Worked Example 4.1 A researcher estimates a three-factor model (returns in percent): $$R_{p}=1.70+1.03R_m-0.23R_z+0.32R_v$$ 1. Compute the fitted portfolio return when $R_m=8\%$, $R_z=2\%$, and $R_v=3\%$. 2. If $R_z$ declines by 1%, what is the estimated change in $R_p$? 3. What is the fitted return when all factors are 0? > **Answer:** (1) Substitute the values: > $$\hat{R}_p=1.70+1.03(8)-0.23(2)+0.32(3)$$ > $$=1.70+8.24-0.46+0.96=10.44\%$$ > (2) Holding other factors fixed, ΔR_p = -0.23 × (-1) = +0.23%. > (3) If all factors are 0, the fitted return equals the intercept: 1.70%. ## **Model fit and joint tests in multiple regression** After estimation, FRM questions often ask you to interpret how well the model fits and whether variables add explanatory power. ### Decomposing variation in Y Define: - Total sum of squares: $$TSS=\sum_{i=1}^n (Y_i-\bar{Y})^2$$ - Residual sum of squares: $$RSS=\sum_{i=1}^n (Y_i-\hat{Y}_i)^2$$ - Explained sum of squares: $$ESS=\sum_{i=1}^n (\hat{Y}_i-\bar{Y})^2$$ These satisfy: $$TSS=ESS+RSS$$ ### R-squared and its limits The coefficient of determination is: $$R^2=\frac{ESS}{TSS}=1-\frac{RSS}{TSS}$$ Interpretation: $R^2$ is the fraction of sample variation in $Y$ explained by the regression model. Pitfalls tested in FRM-style questions: - $R^2$ usually increases when you add regressors, even weak ones. - A high $R^2$ does not prove causality and does not prove you included all relevant variables. - $R^2$ is not directly comparable across models with different dependent variables. ### Adjusted R-squared To penalize adding variables, adjusted $R^2$ is: $$R^2_{\text{adj}}=1-\left(\frac{n-1}{n-k-1}\right)(1-R^2)$$ where $n$ is sample size and $k$ is the number of independent variables. > **Key Term: Adjusted R²** > A version of R² that adjusts for sample size and number of regressors; it can rise or fall when a variable is added. ### The F-test for joint hypotheses When testing more than one restriction (for example, whether several coefficients are jointly zero), use an $F$-test. A common overall test is: $$H_0:\beta_1=\beta_2=\cdots=\beta_k=0\quad\text{vs}\quad H_A:\text{at least one }\beta_j\neq 0$$ One form is: $$F=\frac{ESS/k}{RSS/(n-k-1)}$$ When comparing a **full** model to a **partial** model: $$F=\frac{(RSS_P-RSS_F)/q}{RSS_F/(n-k_F-1)}$$ where $q$ is the number of restrictions (variables removed when moving from full to partial). > **Key Term: F-test** > A one-tailed test used for joint restrictions in regression (for example, whether a group of coefficients adds explanatory power relative to a smaller model). ### **Exam Warning** > Common exam errors in OLS interpretation and testing: > > - Forgetting “holding other variables constant” when interpreting multiple regression slopes (partial effects). > - Using R² alone to judge model quality, or assuming a higher R² means all regressors are important. > - Using the wrong degrees of freedom: simple regression uses n-2, multiple uses n-k-1. > - Treating an intercept as economically meaningful when X=0 is outside the data range. > - Adding a full set of category dummies plus an intercept (dummy variable trap), which creates perfect collinearity. > - Ignoring that heteroskedasticity mainly damages standard errors and hypothesis tests; robust (White) standard errors are then needed. ### Worked Example 5.1 An analyst runs a regression with $n=60$ observations and $k=5$ independent variables. The total sum of squares is $TSS=460$ and the residual sum of squares is $RSS=170$. Compute $R^2$ and adjusted $R^2$. > **Answer:** First compute: > $$R^2=1-\frac{170}{460}=\frac{460-170}{460}=0.630$$ > Then adjust for n and k: > $$R^2_{\text{adj}}=1-\left(\frac{60-1}{60-5-1}\right)(1-0.63)$$ > $$=1-\left(\frac{59}{54}\right)(0.37)=1-0.4041=0.596\ (\text{approx.})$$ ### Worked Example 5.2 A researcher compares a full model (3 regressors) to a partial model (1 regressor) using $n=64$ observations. The residual sums of squares are: - Full model: $RSS_F=6{,}650$ - Partial model: $RSS_P=7{,}140$ Compute the $F$ statistic for adding the two extra variables. > **Answer:** Here q=2 restrictions and n-k_F-1=64-3-1=60. > Numerator: > $$\frac{RSS_P-RSS_F}{q}=\frac{7{,}140-6{,}650}{2}=\frac{490}{2}=245$$ > Denominator: > $$\frac{RSS_F}{n-k_F-1}=\frac{6{,}650}{60}=110.83$$ > So: > $$F=\frac{245}{110.83}=2.21\ (\text{approx.})$$ > You would compare this to the critical F with (2,60) degrees of freedom to decide whether the added variables jointly improve the model. ## **Summary** Simple OLS regression models a dependent variable as a linear function of one explanatory variable plus an error term. The OLS slope and intercept have closed-form expressions based on sample covariance, variance, and means; these are easy to compute and interpret in common FRM settings such as beta estimation, hedge ratio estimation, and performance attribution. Multiple regression extends the same framework to several explanatory variables. Its main added idea is partial slope interpretation: each coefficient measures the effect of its variable holding the others constant. Model fit is summarized by $R^2$ and adjusted $R^2$, while statistical inference relies on $t$-tests for individual coefficients and $F$-tests for joint hypotheses. Finally, correct inference depends on assumptions—especially those affecting standard errors (for example, heteroskedasticity). ## **Key Point Checklist** _This article has covered the following key knowledge points:_ - Simple regression model set-up: $Y=\alpha+\beta X+\varepsilon$ - OLS objective: minimize $RSS=\sum (Y_i-\hat{Y}_i)^2$ - Closed-form estimators: $b=\operatorname{Cov}(X,Y)/\operatorname{Var}(X)$ and $a=\bar{Y}-b\bar{X}$ - Residuals and fitted values: $\hat{Y}_i=a+bX_i$, $e_i=Y_i-\hat{Y}_i$ - Main assumptions for unbiasedness and inference (exogeneity, i.i.d., homoskedasticity, no extreme outliers) - $t$-tests and confidence intervals: $t=(b-B)/s_b$, CI: $b\pm t_c s_b$ - Multiple regression model and partial slope interpretation (“holding other variables constant”) - Additional multiple-regression requirement: no perfect collinearity - Goodness of fit: $TSS=ESS+RSS$, $R^2=ESS/TSS$, adjusted $R^2$ - Joint tests: $F$-tests for groups of coefficients and overall model significance ## **Key Terms and Concepts** - Linear regression - Ordinary least squares (OLS) - Fitted value - Residual - Exogeneity ($E[\varepsilon\mid X]=0$) - Homoskedasticity - Standard error of the regression (SER) - t-statistic and confidence interval - Multiple regression - Partial slope coefficient - Perfect collinearity - Dummy variable - $R^2$ and adjusted $R^2$ - F-test

Regression analysis - Simple and multiple OLS