Who is the Confidence Bootcamp for?

The bootcamp is designed for anyone who wants to improve their experimentation skills. Courses are tailored for data scientists, analysts, engineers, product managers, and leaders — whether you are running your first A/B test or scaling an experimentation program across your organization.

Is the bootcamp free?

Yes, the Confidence Bootcamp is completely free. All 11 courses, 90+ lessons, and resources are available at no cost. You can start learning immediately without creating an account, though signing in lets you track your progress across devices.

The bootcamp covers the full experimentation lifecycle: A/B testing fundamentals, hypothesis formulation, interpreting experiment results, metrics design, sample size calculation, feature flags, and building an experimentation culture. It includes 11 courses with over 90 lessons built by the Confidence team at Spotify.

How long does the bootcamp take to complete?

The full bootcamp takes approximately 20 hours to complete across all 11 courses. Individual courses range from 30 minutes to 3 hours. You can learn at your own pace and pick the courses most relevant to your role.

Do I need prior experience with A/B testing or statistics?

No prior experience is required. The bootcamp starts with foundational courses like Intro to Experimentation and progressively covers more advanced topics like sequential testing and variance reduction. Each course clearly indicates which roles it is designed for.

Who created the Confidence Bootcamp?

The Confidence Bootcamp was created by the Confidence team at Spotify, the same team that builds the experimentation and feature flagging platform used across Spotify. The content reflects real-world experimentation practices used at one of the world's largest digital products.

Welcome to Interpreting experiment results

Interpreting experiment results is an asynchronous, self-paced course that teaches you how to read and understand the results page in Confidence. By the end of this course, you will be able to look at any experiment results page and know exactly what every number, label, and recommendation means, and what to do with that information.

The course is designed to be accessible regardless of your background or role. You do not need a statistics degree to follow along. Where the details matter, this course explains them in plain language and flags where you can dig deeper if you want to.

Some things may look different from other tools you have used. Where Confidence does things its own way, the approach is grounded in years of iteration and original research.

Note

There are quiz questions throughout the course to help you check your understanding of the material. Complete each lesson's questions to track your progress.

Before you begin

This course works best if you have run at least one experiment in Confidence, or have followed the A/B test quickstart. Having a concrete experiment in mind as you go through the lessons will help the concepts click.

Lessons

This course consists of the following lessons:

Lesson 1: The anatomy of the results page

Get oriented on the three sections of the results page and understand the basic logic connecting them.

Not completed

Lesson 2: The Spotlight

Understand the overall recommendation (Ship, Continue, End, or Abort) and what drives each one.

Not completed

Lesson 3: Means and relative effects

Understand what the control variant and treatment variant means represent, and why effects are shown as relative percentages.

Not completed

Lesson 4: Confidence intervals and precision

Learn what confidence intervals are, how to read them, and why their width tells you how precisely the effect has been measured.

Not completed

Lesson 5: Significance for success metrics

Understand what 'significant' and 'not significant' mean for success metrics, and how the CI position determines the status.

Not completed

Lesson 6: Guardrail metrics and NIMs

Learn how guardrail metric status labels work, what a non-inferiority margin is, and why it gives stronger evidence of safety.

Not completed

Lesson 7: Health checks and the SRM

Learn how Confidence verifies that your experiment is trustworthy, and what to do when a health check fails.

Not completed

Lesson 8: Variance reduction

Understand why the means shown in results may differ slightly from raw averages, and how to interpret them correctly.

Not completed

Lesson 9: Sequential and non-sequential tests

Learn when you can trust the results you see, and what your choice of evaluation strategy means for your experiment.

Not completed

Lesson 10: Exploratory analysis

Learn how to use explorations to learn more from your experiment without drawing false conclusions.

Not completed

Lesson 11: The winner's curse

Learn about the winner's curse, why significant results from underpowered experiments tend to overestimate the true effect, and how to use confidence interval precision as a practical safeguard.

Not completed