Null vs Alternative Hypothesis
Four Steps in Data-Driven Decision Making:
- Formulate a hypothesis
- Find the right test
- Execute the test
- Make a decision
A hypothesis is an idea that can be tested.
Null Hypotheses (N0) – the hypothesis to be tested (i.e. DS salaries average $114,000).
Alternative Hypothesis (H1 or HA) – Everything else (i.e. DS salaries do not average $114,000).
In general, the researcher is trying to reject the null hypothesis. If he cannot, then it must be true. Another way to look at this is that the Null is the present state of affairs, while the alternative is the researcher’s personal opinion.
Rejection Region and Significance Level
Significance Level (α) is the probability of rejecting the null hypothesis, if it is true. Typical values for alpha are 0.01, 0.05, and 0.1. Alpha is selected based on the problem at hand and the constraints needed for that problem. In simpler terms, the significance level determines the threshold at which we consider the evidence against a null hypothesis to be strong enough to reject it in favor of an alternative hypothesis.
The Z Test
- x hat is the sample mean
- µ is the hypothesized mean
- s / sqrt(n) is the standard error
This scales the mean. A value of 0 means that the null hypothesis is proven. Large negative or positive values (usually +/- 1.96) correspond to the rejection region.
How does Hypothesis Testing Work?
- Calculate the statistic (e.g. x hat)
- Scale it with the Z calculation
- Check if Z is in the rejection region
This is the case for a Two-Sided test where $114,000 is an exact value, and there could be values lower or higher than the null.
There are also One-Sided tests (i.e. DS average salary is more than $125,000). Here, the rejection region is salaries at $125,000 or lower. If α = 0.05, the limit is -1.645.
Type I Error and Type II Error
Type I Error is when you reject a true null hypothesis. This is also called a False Positive. The probability of making this error is α, the level of significance.
Type II Error is when you accept a false null hypothesis. This is also called a False Negative. The probability of making this error is β, which is impacted by sample size (n) and variance (σ).
The probability of rejecting a false null hypothesis, which is the researcher’s goal, is called the Power of the Test and has the probability of 1 – β.
| H0 is True | H0 is False | |
| Accept | OK | Type II Error (False Negative) |
| Reject | Type I Error (False Positive) | OK |
Test for the Mean (Population Variance Known)
Use the Z Test (aka the Z Score).
Decision Rule: Reject if the absolute value of the Z-score > positive critical value (z).
p-Value
p-value is the smallest level of significance at which we can still reject the null hypothesis, given the observed sample statistic.
Rule: You should reject the null hypothesis if p-value < α.
p-value is usually calculated by software.
Find the p-value manually
Use the Z score to find the value of z on the z table. Once you know z:
For a One-Sided problem, p value is the number from the table.
For a Two-Sided problem, p value is the number from the table times 2.
Test for the Mean (Population Variance Unknown)
Rule: Accept if the absolute value of the T score < critical value t.
If we cannot reject a test at 0.05, we cannot reject it at smaller levels either.