Who is the Confidence Bootcamp for?

The bootcamp is designed for anyone who wants to improve their experimentation skills. Courses are tailored for data scientists, analysts, engineers, product managers, and leaders — whether you are running your first A/B test or scaling an experimentation program across your organization.

Is the bootcamp free?

Yes, the Confidence Bootcamp is completely free. All 11 courses, 90+ lessons, and resources are available at no cost. You can start learning immediately without creating an account, though signing in lets you track your progress across devices.

The bootcamp covers the full experimentation lifecycle: A/B testing fundamentals, hypothesis formulation, interpreting experiment results, metrics design, sample size calculation, feature flags, and building an experimentation culture. It includes 11 courses with over 90 lessons built by the Confidence team at Spotify.

How long does the bootcamp take to complete?

The full bootcamp takes approximately 20 hours to complete across all 11 courses. Individual courses range from 30 minutes to 3 hours. You can learn at your own pace and pick the courses most relevant to your role.

Do I need prior experience with A/B testing or statistics?

No prior experience is required. The bootcamp starts with foundational courses like Intro to Experimentation and progressively covers more advanced topics like sequential testing and variance reduction. Each course clearly indicates which roles it is designed for.

Who created the Confidence Bootcamp?

The Confidence Bootcamp was created by the Confidence team at Spotify, the same team that builds the experimentation and feature flagging platform used across Spotify. The content reflects real-world experimentation practices used at one of the world's largest digital products.

Welcome to A primer on hypothesis testing

A primer on hypothesis testing is an asynchronous, self-paced course that teaches you the fundamental concepts of hypothesis testing in the context of experimentation. This course focuses on building intuition for how hypothesis tests work and why they are crucial for making decisions in experiments.

In this course, you'll learn how hypothesis testing helps manage uncertainty in experimental data, understand the relationship between what you observe in a sample and what is true for the full population, and learn how to interpret statistical significance and p-values.

Note

There are quiz questions throughout the course to help you check your understanding of the material. Complete each lesson's questions to track your progress.

Before you begin

Before you start this course, you should have a basic understanding of what it means to run an experiment end-to-end. In our experience, this helps you internalize the concepts in our courses.

In Confidence

If you haven't already, go through the A/B test quickstart to get hands-on experience running an experiment in Confidence.

Lessons

This course consists of the following lessons:

Lesson 1: What is hypothesis testing?

Learn the basics of hypothesis testing and how it helps handle uncertainty in experimental data.

Not completed

Lesson 2: True versus estimated difference in means

Understand the difference between population parameters and sample statistics in hypothesis testing.

Not completed

Lesson 3: The sampling distribution

Learn about sampling distributions and the Central Limit Theorem's role in hypothesis testing.

Not completed

Lesson 4: P-values and rejecting the null hypothesis

Learn about the interpretation of p-values and learn how to make decisions based on statistical significance.

Not completed

Lesson 5: False positive rate and alpha

Learn about the false positive rate and how it relates to the alpha parameter.

Not completed

Lesson 6: True positive rate and power

Learn about the true positive rate and how it relates to the power parameter.

Not completed

Welcome to A primer on hypothesis testing

Note

There are quiz questions throughout the course to help you check your understanding of the material. Complete each lesson's questions to track your progress.

Before you begin

Before you start this course, you should have a basic understanding of what it means to run an experiment end-to-end. In our experience, this helps you internalize the concepts in our courses.

In Confidence

If you haven't already, go through the A/B test quickstart to get hands-on experience running an experiment in Confidence.

Lessons

This course consists of the following lessons:

Lesson 1: What is hypothesis testing?

Learn the basics of hypothesis testing and how it helps handle uncertainty in experimental data.

Not completed

Lesson 2: True versus estimated difference in means

Understand the difference between population parameters and sample statistics in hypothesis testing.

Not completed

Lesson 3: The sampling distribution

Learn about sampling distributions and the Central Limit Theorem's role in hypothesis testing.

Not completed

Lesson 4: P-values and rejecting the null hypothesis

Learn about the interpretation of p-values and learn how to make decisions based on statistical significance.

Not completed

Lesson 5: False positive rate and alpha

Learn about the false positive rate and how it relates to the alpha parameter.

Not completed

Lesson 6: True positive rate and power

Learn about the true positive rate and how it relates to the power parameter.

Not completed