Learning outcomes
- Explain the key idea in your own words.
- Apply the concept to a realistic but fictional research scenario.
- Identify assumptions and limits before interpreting numbers.
Conceptual lesson
The null hypothesis H0 is a specified model claim, often no difference or no association; the alternative H1 is another claim. A p-value is the probability, assuming H0 and the test assumptions, of a statistic at least as extreme as the observed result.
A p-value is not the probability that H0 is true and not a direct measure of practical importance. A 0.05 threshold is a decision convention that does not make p=0.049 and p=0.051 scientifically opposite realities. Running many tests increases false-positive risk.
Interpret tests alongside effect sizes, confidence intervals, assumptions, study design and context. Low power may miss meaningful effects; “not statistically significant” does not establish that the effect is zero.
Worked research example
A result of p=0.04 does not mean that H0 has a 4% chance of being true. It means that under H0 and the model assumptions, the chance of seeing a result this extreme or more extreme is 4%.
Common reasoning trap
Do not equate “statistically significant” with large, important, causal or certainly replicable.
Practice tasks
- 1. Define a p-value as a conditional probability.
- 2. Identify two incorrect conclusions someone might draw from p=0.06.
- 3. Report a hypothetical result using an effect size and interval alongside the p-value.
Explain your reasoning, not just the numerical result.
Optional R practice
Download the fictional CSV into your working folder, then run the script below or download the .R file. No additional packages are needed.
Download CSV · Download R lab · Module 0 refresher
# Statistics Lab / ferdo — synthetic data (no real student records)
# Save stats_synthetic.csv in your working directory.
d <- read.csv("stats_synthetic.csv", stringsAsFactors = TRUE)
stopifnot(nrow(d) == 36, all(c("score_pre","score_post","group") %in% names(d)))
# Groups are observational labels: differences cannot establish a treatment effect.
set.seed(2026)
null_means <- replicate(5000, mean(rnorm(20, mean=0, sd=1)))
observed <- .5
mean(abs(null_means) >= abs(observed))
hist(null_means, main="Simulated null sampling distribution")
abline(v=c(-observed,observed), lty=2)Stretch: mathematical track
Discuss why a family of many significance tests can inflate the chance of at least one false positive.
Exit ticket
Write one evidence-bounded claim and one limitation about this week's example.
Student workbook for this module
Four written problems, marking guide and a formative four-question quiz.
Optional mathematical derivations
Extend this topic with worked mathematics, written exercises and base R checks.