## **Learning Outcomes**

This article explains how hypothesis tests turn sample evidence into disciplined decisions. After reading, you will be able to:

- Calculate and interpret common test statistics (z, t) using standard errors.
- Interpret p-values correctly for one- and two-tailed tests and connect them to significance levels.
- Explain power, Type I and Type II errors, and how sample size and test size affect them.

## **GARP FRM Part 1 Syllabus**

For the GARP FRM Part 1 exam, you are expected to understand hypothesis testing mechanics with a focus on the following syllabus points:

- Construct null and alternative hypotheses; choose one-sided versus two-sided tests (LO 17.a, 17.b).
- Relate Type I and Type II errors to significance level and power (LO 17.c).
- Interpret p-values and connect hypothesis tests to confidence intervals (LO 17.d, 17.e, 17.f).

## **Test Your Knowledge**

_Attempt these questions before reading this article. If you find some difficult or cannot remember the answers, remember to look more closely at that area during your revision._

1. A risk analyst reports a p-value of 0.03 for a two-tailed test. Which statement is most accurate?  
   a) There is a 3% probability that the null hypothesis is true  
   b) The null would be rejected at the 5% significance level  
   c) The null must be rejected at the 1% significance level  
   d) The probability of a Type II error is 3%

2. Calculation: You test $H_0:\mu=0$ vs. $H_A:\mu>0$ using $n=36$, known $\sigma=12$, and sample mean $\bar{x}=3$. Compute the z-statistic.

3. If you reduce the significance level from 5% to 1% (holding everything else fixed), what happens to power, and why?

4. True/False (brief explanation required): It is correct to say “accept $H_0$” when the test statistic is not in the rejection region.

## **What Do You Know Already?**

- Can you explain what “standard error” measures and why it appears in every test statistic?
- Can you compute a two-tailed p-value from a z-statistic using symmetry?
- Can you describe the trade-off between Type I error control (size) and Type II error control (power)?

Use these prompts to gauge your starting point; then focus your revision where your answers were least confident.

## **Introduction**

Risk management is decision-making under uncertainty: you observe noisy data (returns, spreads, losses, defaults) and must decide whether the evidence supports a claim about the true population (a mean, a variance, a regression coefficient, a model assumption). Hypothesis testing provides a disciplined way to do this. It forces you to state what you assume, what evidence would contradict that assumption, and how often you are willing to be wrong in a specific way.

For FRM candidates, the recurring challenge is interpretation rather than arithmetic. You can often compute a z- or t-statistic quickly, but the exam frequently tests whether you understand what a p-value actually means, why power matters, and how changing the significance level or sample size changes the error rates.

> **Key Term: Hypothesis test**  
> A statistical procedure that uses a sample to decide whether to reject a statement about a population parameter, using a pre-specified error probability.

## **Hypothesis testing as a decision rule**

A hypothesis is a statement about a population parameter (such as a mean $\mu$ or a regression slope $\beta$). A hypothesis test is structured around two competing statements:

- The **null hypothesis** $H_0$, the statement assumed true for the purpose of constructing the test.
- The **alternative hypothesis** $H_A$, the statement you conclude is supported when evidence against $H_0$ is strong enough.

The FRM convention (and exam favorite) is:

- **The null always includes an equality condition** ($=$, $\le$, or $\ge$).
- The alternative carries the “direction” of the claim.

Examples for a population mean:

- Two-sided: $H_0:\mu=\mu_0$ vs. $H_A:\mu\neq\mu_0$
- Upper tail: $H_0:\mu\le \mu_0$ vs. $H_A:\mu>\mu_0$
- Lower tail: $H_0:\mu\ge \mu_0$ vs. $H_A:\mu<\mu_0$

A complete test also requires five operational ingredients:

1. **Test statistic** computed from sample data.
2. **Significance level** $\alpha$ (also called **test size**) that controls Type I error probability.
3. **Sampling distribution** of the test statistic under $H_0$ (z, t, $\chi^2$, F, etc.).
4. **Critical value(s)** determined from that distribution and $\alpha$.
5. **Decision rule**: reject $H_0$ if the test statistic falls in the rejection region.

> **Key Term: Significance level ($\alpha$)**  
> The probability of rejecting $H_0$ when $H_0$ is actually true (the Type I error probability).

### One-tailed versus two-tailed tests

The choice is determined by the alternative hypothesis:

- **One-tailed test**: rejection region is only in one tail (upper or lower).
- **Two-tailed test**: rejection regions are split across both tails.

This choice matters for interpretation and power:

- A two-tailed test is appropriate when deviations in either direction are meaningful (e.g., “mean return differs from zero”).
- A one-tailed test is appropriate when only one direction supports the claim (e.g., “mean return is greater than the risk-free rate”).

### Worked Example 1.1

A desk analyst claims the mean daily P&L of a strategy is positive. Using $n=36$ days, $\bar{x}=3$ (in \$thousands), and known $\sigma=12$ (in \$thousands), test at $\alpha=5\%$:

$$H_0:\mu\le 0 \quad\text{vs.}\quad H_A:\mu>0$$

> **Answer:**  
> Compute the standard error of the sample mean:  
> $$SE(\bar{x})=\frac{\sigma}{\sqrt{n}}=\frac{12}{\sqrt{36}}=\frac{12}{6}=2$$  
> Compute the z-statistic:  
> $$z=\frac{\bar{x}-\mu_0}{\sigma/\sqrt{n}}=\frac{3-0}{2}=1.50$$  
> For a one-tailed test at $\alpha=0.05$, the critical value is approximately $z_{0.95}=1.65$.
>
> Decision: since $1.50<1.65$, the statistic is not in the rejection region. **Fail to reject $H_0$**.  
> Interpretation: the sample evidence is not strong enough at the 5% level to conclude the mean is positive.

## **Test statistics and standard errors**

Almost every FRM Part 1 hypothesis test has the same structure:

$$\text{test statistic}=\frac{\text{estimate} - \text{hypothesized value}}{\text{standard error of the estimate}}$$

> **Key Term: Test statistic**  
> A standardized measure (often a z- or t-value) that converts “distance from the null” into units of sampling variability, so it can be compared to a known reference distribution.

### Why the denominator is a standard error

A **standard deviation** describes dispersion in the data. A **standard error** describes dispersion in an estimator across repeated samples.

For the sample mean, if the population standard deviation is $\sigma$ and the sample size is $n$, then:

$$SE(\bar{X})=\frac{\sigma}{\sqrt{n}}$$

As $n$ increases, $SE(\bar{X})$ falls, so the same “economic difference” $\bar{x}-\mu_0$ produces a larger test statistic. This is one reason large datasets often generate very small p-values.

> **Key Term: Standard error**  
> The standard deviation of an estimator’s sampling distribution; it measures estimator uncertainty and typically shrinks as sample size increases.

### z-statistics versus t-statistics for a mean

For tests about a population mean:

- Use the **z-test** when the population variance is known (and commonly when $n$ is large).
- Use the **t-test** when the population variance is unknown and must be estimated from the sample.

Key formulas:

- Known variance:
  $$z=\frac{\bar{x}-\mu_0}{\sigma/\sqrt{n}}$$

- Unknown variance:
  $$t=\frac{\bar{x}-\mu_0}{s/\sqrt{n}}$$  
  where $s$ is the sample standard deviation and the t-statistic has $n-1$ degrees of freedom.

### Test statistics in regression (why FRM still cares here)

In OLS regression, individual coefficients are tested with a t-statistic of the same “estimate minus hypothesized value over standard error” form. For a coefficient $b_j$:

$$t=\frac{b_j-B_j}{s_{b_j}}$$

Regression output typically reports:

- the coefficient estimate,
- its standard error,
- the t-statistic,
- the p-value.

Even though this article is not a full regression chapter, the interpretation rules for p-values and confidence intervals are the same as for mean tests.

### Worked Example 2.1

A single-factor regression estimates a slope coefficient $b=0.76$ with standard error $s_b=0.33$ using $n=46$ observations. Test $H_0:\beta=0$ vs. $H_A:\beta\neq 0$ at 5% significance, with critical value $t_c=2.02$.

> **Answer:**  
> Compute the t-statistic:  
> $$t=\frac{0.76-0}{0.33}=2.30$$  
> Compare to the critical value (two-tailed): reject if $|t|>2.02$.
>
> Decision: $|2.30|>2.02$, so **reject $H_0$**.  
> Interpretation: the slope is statistically different from zero at the 5% level.

## **p-values and statistical evidence**

A critical-value approach answers: “Does the statistic fall into the rejection region for my chosen $\alpha$?”

A p-value approach answers: “How extreme is my sample evidence if $H_0$ is true?”

Both methods produce the same reject/fail-to-reject decision when used consistently.

> **Key Term: p-value**  
> The probability of obtaining a test statistic that would lead to rejection of $H_0$, assuming $H_0$ is true; equivalently, the smallest significance level at which $H_0$ would be rejected.

### How to compute p-values (z-test intuition)

Let $z_{\text{obs}}$ be the computed value. Under $H_0$, a z-statistic follows a standard normal distribution.

- Upper-tail test ($H_A:\mu>\mu_0$):
  $$p=P(Z\ge z_{\text{obs}})$$

- Lower-tail test ($H_A:\mu<\mu_0$):
  $$p=P(Z\le z_{\text{obs}})$$

- Two-tailed test ($H_A:\mu\neq\mu_0$):
  $$p=2P(Z\ge |z_{\text{obs}}|)$$

Decision rule using p-values:

- If $p<\alpha$, reject $H_0$.
- If $p\ge\alpha$, fail to reject $H_0$.

### Worked Example 3.1

A two-tailed z-test produces $z_{\text{obs}}=2.3$. Compute the p-value and decide whether to reject at 5% and at 1%.

> **Answer:**  
> For a standard normal variable, the cumulative probability to the left of 2.3 is about 0.9893.  
> Upper-tail probability:  
> $$P(Z>2.3)=1-0.9893=0.0107$$  
> Two-tailed p-value:  
> $$p=2\times 0.0107=0.0214=2.14\%$$  
> Decisions:
>
> - At $\alpha=5\%$: $p<\alpha$ so **reject $H_0$**.
> - At $\alpha=1\%$: $p>\alpha$ so **fail to reject $H_0$**.

### **Exam Warning**

> Common p-value mistakes cost marks:
>
> - A p-value is **not** the probability that $H_0$ is true.
> - “Fail to reject $H_0$” is **not** the same as “$H_0$ is correct.” You did not obtain strong evidence against it at the chosen $\alpha$.
> - The p-value depends on whether the test is one- or two-tailed. Using a two-tailed p-value for a one-tailed alternative (or vice versa) is a frequent exam error.

## **Power and the trade-off between errors**

Any hypothesis test can be wrong in two distinct ways:

- **Type I error**: reject $H_0$ when $H_0$ is true. Its probability is $\alpha$.
- **Type II error**: fail to reject $H_0$ when $H_0$ is false. Its probability is often denoted $\beta$.

Power is the complement of the Type II error probability.

> **Key Term: Power**  
> The probability of correctly rejecting $H_0$ when $H_0$ is false; $\text{Power}=1-\beta$.

### Why power matters in risk management

Risk managers often test statements like:

- “This strategy has positive alpha.”
- “This hedging program reduces variance.”
- “A model change improves forecast accuracy.”

A low-power test may fail to detect a real effect, leading to poor decisions (for example, discarding a genuinely useful hedge because the sample was too small to show significance).

### What determines power

Power depends on three drivers (and the FRM exam tests these directional relationships):

1. **Sample size ($n$)**: larger $n$ reduces standard errors and generally increases power.
2. **Effect size**: the “distance” between the true parameter and the null value. Larger differences are easier to detect.
3. **Significance level ($\alpha$)**: lowering $\alpha$ (stricter Type I error control) raises critical values and typically reduces power.

There is a trade-off: for a fixed sample size and variability, making Type I errors rarer usually makes Type II errors more likely.

### Power calculation for a one-sided z-test

Consider $H_0:\mu\le\mu_0$ vs. $H_A:\mu>\mu_0$ with known $\sigma$.

- Reject when:
  $$z=\frac{\bar{X}-\mu_0}{\sigma/\sqrt{n}} > z_{1-\alpha}$$

If the true mean is $\mu_1>\mu_0$, then under the true data-generating process the statistic behaves like a normal variable shifted to the right. Power is:

$$\text{Power}(\mu_1)=P_{\mu=\mu_1}\left(\frac{\bar{X}-\mu_0}{\sigma/\sqrt{n}} > z_{1-\alpha}\right)$$

### Worked Example 4.1

Test $H_0:\mu\le 0$ vs. $H_A:\mu>0$ at $\alpha=5\%$ using known $\sigma=1$.

1. Compute power when $n=25$ if the true mean is $\mu_1=0.4$.
2. Recompute power when $n=100$ (same $\mu_1$, $\sigma$, and $\alpha$).

> **Answer:**  
> For a one-tailed 5% test, $z_{1-\alpha}\approx 1.645$.
>
> **1) When $n=25$**  
> Standard error: $SE=1/\sqrt{25}=0.2$  
> Under $\mu_1=0.4$, the z-statistic is centered at:  
> $$E[z]=\frac{\mu_1-\mu_0}{SE}=\frac{0.4-0}{0.2}=2.0$$  
> The statistic has standard deviation 1.
>
> Power:  
> $$\text{Power}=P(Z>1.645)\ \text{where}\ Z\sim N(2,1)$$  
> Standardize:  
> $$P\left(N(0,1)>\frac{1.645-2}{1}\right)=P(N(0,1)>-0.355)=\Phi(0.355)\approx 0.638$$  
> So power is about **63.8%**, and Type II error probability is about **36.2%**.
>
> **2) When $n=100$**  
> Standard error: $SE=1/\sqrt{100}=0.1$  
> New center:  
> $$E[z]=\frac{0.4}{0.1}=4.0$$  
> Power:  
> $$P(Z>1.645)\ \text{where}\ Z\sim N(4,1)=P(N(0,1)>1.645-4)$$  
> $$=P(N(0,1)>-2.355)=\Phi(2.355)\approx 0.991$$  
> Power is about **99.1%**. Increasing sample size sharply improves power.

## **Confidence intervals as an equivalent test**

Confidence intervals (CIs) and hypothesis tests are linked through the same critical values.

A two-sided $(1-\alpha)$ CI for a parameter $\theta$ is:

$$\hat{\theta} \pm (\text{critical value})\times SE(\hat{\theta})$$

For a mean (z-case):

$$\bar{x} \pm z_{1-\alpha/2}\frac{\sigma}{\sqrt{n}}$$

Decision equivalence (two-tailed):

- If the hypothesized value $\theta_0$ lies **outside** the $(1-\alpha)$ CI, reject $H_0:\theta=\theta_0$.
- If $\theta_0$ lies **inside** the CI, fail to reject.

One-sided confidence intervals align with one-tailed tests:

- Upper-tail test ($H_A:\theta>\theta_0$) corresponds to a lower-bounded interval:
  $$\left[\hat{\theta}-z_{1-\alpha}SE(\hat{\theta}),\ \infty\right)$$
- Lower-tail test corresponds to an upper-bounded interval:
  $$\left(-\infty,\ \hat{\theta}+z_{1-\alpha}SE(\hat{\theta})\right]$$

### Worked Example 5.1

Using Worked Example 1.1 data ($n=36$, $\bar{x}=3$, $\sigma=12$ in \$thousands), construct a 95% one-sided lower confidence bound for $\mu$ (consistent with $H_A:\mu>0$). Does it support rejecting at $\alpha=5\%$?

> **Answer:**  
> Standard error: $SE=2$ (from Worked Example 1.1).  
> For a 95% lower confidence bound, use $z*{0.95}\approx 1.65$:  
> $$\mu \ge \bar{x}-z*{0.95}SE = 3 - 1.65(2)=3-3.30=-0.30$$  
> The 95% lower bound is about $-0.30$ (in \$thousands), which is below 0.  
> Since 0 is not excluded, this matches the earlier result: **fail to reject** at 5%.

## **Practical considerations for risk managers**

### Multiple testing and false discoveries

Testing many hypotheses on the same dataset increases the chance of at least one false rejection. If you run 20 independent tests at $\alpha=5\%$, the expected number of false positives is about $20\times 0.05=1$, and the probability of at least one false positive can be much larger than 5%.

> **Key Term: Multiple testing**  
> Running many hypothesis tests on the same data, which increases the overall probability of a Type I error beyond the nominal $\alpha$ of any single test.

In trading strategy research, this appears as “strategy mining”: trying many signals until one looks statistically significant. In model validation, it appears when many performance metrics are checked and only the best-looking ones are reported.

Practical controls include:

- Pre-specifying a small set of hypotheses (before seeing results).
- Adjusting significance thresholds (for example, Bonferroni-style corrections).
- Using out-of-sample testing to reduce selection bias.

### Statistical significance versus economic significance

With very large samples, standard errors can become very small. That can make p-values tiny even when the effect is economically trivial (for example, a coefficient that changes expected return by 1 basis point per year). In risk management, decisions should consider both:

- statistical evidence (p-values, confidence intervals), and
- business impact (P&L, hedging error reduction, capital effects, operational cost).

### Interpreting “no rejection” carefully

Failing to reject may mean:

- the null is close to true, or
- the test has low power (sample too small, high volatility, effect too small to detect).

On the exam, this often appears as a qualitative question: “Which change increases power?” The safe answers are usually “increase sample size” or “use a one-tailed test if direction is justified,” while noting that lowering $\alpha$ reduces power.

## **Summary**

Test statistics, p-values, and power are three views of the same decision system. The test statistic standardizes sample evidence using a standard error so it can be compared to a known reference distribution. The p-value expresses how extreme that evidence would be if the null were true and can be compared directly to a chosen significance level.

Power complements this by describing performance when the null is false: it is the probability the test detects a real effect. Power rises with sample size and effect size and typically falls as you tighten Type I error control by lowering $\alpha$. In practice, risk managers must also guard against multiple testing and must distinguish statistical significance from economic relevance.

## **Key Point Checklist**

_This article has covered the following key knowledge points:_

- A hypothesis test requires $H_0$, $H_A$, a test statistic, a significance level $\alpha$, critical values, and a decision rule.
- The null hypothesis includes an equality condition; one- versus two-tailed tests follow from $H_A$.
- A generic test statistic is $\frac{\text{estimate}-\text{hypothesized value}}{SE(\text{estimate})}$.
- The p-value is computed under $H_0$ and is the smallest $\alpha$ at which $H_0$ would be rejected.
- Type I error probability equals $\alpha$; power equals $1-\beta$ where $\beta$ is Type II error probability.
- Increasing $n$ generally increases power; decreasing $\alpha$ generally decreases power.
- Hypothesis tests and confidence intervals are equivalent decision tools when constructed at consistent confidence/significance levels.
- Multiple testing increases the overall chance of false positives and can bias reported results.

## **Key Terms and Concepts**

- Alternative hypothesis ($H_A$)
- Confidence interval
- Critical value
- Multiple testing
- Null hypothesis ($H_0$)
- p-value
- Power ($1-\beta$)
- Significance level (test size, $\alpha$)
- Standard error
- Test statistic
- Type I error
- Type II error
- One-tailed test
- Two-tailed test


Hypothesis testing - Test statistics, p-values, and power