Welcome to Intro to experimentation
Intro to experimentation is an asynchronous, self-paced course where you dive deeper into experimentation. While experimentation is a broad topic, this course focuses on teaching you the basics of A/B testing. Much of what you learn in the course also applies to other types of experiments, like rollouts.
In the course, you learn the key concepts of A/B testing and how to set up, run, and monitor an A/B test—interchangeably referred to as an experiment—using Confidence. The focus of the course is on A/B tests, but much of what you learn in the course also applies to rollouts and other types of experiments.
Before you start this course, you should have a basic understanding of what it means to run an experiment end-to-end. In our experience, this helps you internalize the concepts in our courses.
If you haven't already, go through the A/B test quickstart to get hands-on experience running an experiment in Confidence.
There are quiz questions throughout the course to help you check your understanding of the material.
Lessons
This course consists of the following lessons:
Lesson 1: Why you should experiment
Learn about the benefits of experimentation and how it can help you make better decisions.
Lesson 2: Experiment hypothesis
Learn how to plan your experiment and craft a solid product hypothesis.
Lesson 3: Success and guardrail metrics
Learn about the roles of success and guardrail metrics in an experiment.
Lesson 4: Success metrics
Learn how to select metrics and how to configure the sensitivity to detect effects.
Lesson 5: Set up your experiment
Learn how to set up an experiment, and how to collaborate with your team in Confidence.
Lesson 6: Calculation frequency
Learn about the options for when to calculate results and how to choose between them.
Lesson 7: Target audience
Learn how to select who to target with your experiment.
Lesson 8: Sample size
Learn how to calculate the sample size you need to detect the effects you are interested in.
Lesson 9: Quality assurance
Learn about the steps you can take to ensure that your experiment works as intended before you launch.
Lesson 10: Run your experiment
Learn about what to keep track of while your experiment is live.
Lesson 11: Evaluate your experiment and make a decision
Learn how to interpret the results of your experiment and decide the next steps for the product you are iterating on.
Lesson 12: Choosing between A/B tests and rollouts
Learn how to choose between A/B tests and rollouts and when to use both.