Welcome to A primer on hypothesis testing
A primer on hypothesis testing is an asynchronous, self-paced course that teaches you the fundamental concepts of hypothesis testing in the context of experimentation. This course focuses on building intuition for how hypothesis tests work and why they are crucial for making decisions in experiments.
In this course, you'll learn how hypothesis testing helps manage uncertainty in experimental data, understand the relationship between what you observe in a sample and what is true for the full population, and learn how to interpret statistical significance and p-values.
There are quiz questions throughout the course to help you check your understanding of the material. Complete each lesson's questions to track your progress.
Before you begin
Before you start this course, you should have a basic understanding of what it means to run an experiment end-to-end. In our experience, this helps you internalize the concepts in our courses.
If you haven't already, go through the A/B test quickstart to get hands-on experience running an experiment in Confidence.
Lessons
This course consists of the following lessons:
Lesson 1: What is hypothesis testing?
Learn the basics of hypothesis testing and how it helps handle uncertainty in experimental data.
Lesson 2: True versus estimated difference in means
Understand the difference between population parameters and sample statistics in hypothesis testing.
Lesson 3: The sampling distribution
Learn about sampling distributions and the Central Limit Theorem's role in hypothesis testing.
Lesson 4: P-values and rejecting the null hypothesis
Learn about the interpretation of p-values and learn how to make decisions based on statistical significance.
Lesson 5: False positive rate and alpha
Learn about the false positive rate and how it relates to the alpha parameter.
Lesson 6: True positive rate and power
Learn about the true positive rate and how it relates to the power parameter.