Who is the Confidence Bootcamp for?

The bootcamp is designed for anyone who wants to improve their experimentation skills. Courses are tailored for data scientists, analysts, engineers, product managers, and leaders — whether you are running your first A/B test or scaling an experimentation program across your organization.

Is the bootcamp free?

Yes, the Confidence Bootcamp is completely free. All 11 courses, 90+ lessons, and resources are available at no cost. You can start learning immediately without creating an account, though signing in lets you track your progress across devices.

The bootcamp covers the full experimentation lifecycle: A/B testing fundamentals, hypothesis formulation, interpreting experiment results, metrics design, sample size calculation, feature flags, and building an experimentation culture. It includes 11 courses with over 90 lessons built by the Confidence team at Spotify.

How long does the bootcamp take to complete?

The full bootcamp takes approximately 20 hours to complete across all 11 courses. Individual courses range from 30 minutes to 3 hours. You can learn at your own pace and pick the courses most relevant to your role.

Do I need prior experience with A/B testing or statistics?

No prior experience is required. The bootcamp starts with foundational courses like Intro to Experimentation and progressively covers more advanced topics like sequential testing and variance reduction. Each course clearly indicates which roles it is designed for.

Who created the Confidence Bootcamp?

The Confidence Bootcamp was created by the Confidence team at Spotify, the same team that builds the experimentation and feature flagging platform used across Spotify. The content reflects real-world experimentation practices used at one of the world's largest digital products.

Welcome to Sample size calculation — level III

Sample size calculation — level III is an asynchronous, interactive, self-paced course that teaches you the fundamental concepts of how to calculate the required sample size for experiments. This course focuses on building intuition for why you need to plan the sample size before the experiment and what settings and aspects that affect the required sample size.

In this course, you'll learn about some of the more mathematical aspects of sample size calculation. You learn about variance reduction and how it reduces the required sample size. You also get to build intuition for how the sample size calculation differs between binary and continuous metrics, and how the treatment group proportions affect the required sample size. Finally, you get to play around with the complete sample size calculator, accounting for all the aspects covered in all three sample size courses.

Note

There are quiz questions throughout the course to help you check your understanding of the material. Complete each lesson's questions to track your progress.

Before you begin

This is the third and final level of the Sample Size Calculation course series. If you haven't already, complete the previous levels before starting this course:

Lessons

This course consists of the following lessons:

Lesson 1: Binary metrics

Understand how sample size calculations differ between binary and continuous metrics.

Not completed

Lesson 2: Treatment group proportions

Build intuition for why the relative sizes of the treatment groups affect the required sample size.

Not completed

Lesson 3: Variance Reduction

Learn why variance reduction might as well be called sample size reduction.

Not completed

Lesson 4: Sequential testing and sample size

Learn how sequential tests trade off extra data for the ability to monitor results during an experiment.

Not completed

Lesson 5: Sample size playground

Build intuition for sample size calculation by playing with different settings in the playground.

Not completed

Welcome to Sample size calculation — level III

Note

There are quiz questions throughout the course to help you check your understanding of the material. Complete each lesson's questions to track your progress.

Before you begin

This is the third and final level of the Sample Size Calculation course series. If you haven't already, complete the previous levels before starting this course:

Lessons

This course consists of the following lessons:

Lesson 1: Binary metrics

Understand how sample size calculations differ between binary and continuous metrics.

Not completed

Lesson 2: Treatment group proportions

Build intuition for why the relative sizes of the treatment groups affect the required sample size.

Not completed

Lesson 3: Variance Reduction

Learn why variance reduction might as well be called sample size reduction.

Not completed

Lesson 4: Sequential testing and sample size

Learn how sequential tests trade off extra data for the ability to monitor results during an experiment.

Not completed

Lesson 5: Sample size playground

Build intuition for sample size calculation by playing with different settings in the playground.

Not completed