Who is the Confidence Bootcamp for?

The bootcamp is designed for anyone who wants to improve their experimentation skills. Courses are tailored for data scientists, analysts, engineers, product managers, and leaders — whether you are running your first A/B test or scaling an experimentation program across your organization.

Is the bootcamp free?

Yes, the Confidence Bootcamp is completely free. All 11 courses, 90+ lessons, and resources are available at no cost. You can start learning immediately without creating an account, though signing in lets you track your progress across devices.

The bootcamp covers the full experimentation lifecycle: A/B testing fundamentals, hypothesis formulation, interpreting experiment results, metrics design, sample size calculation, feature flags, and building an experimentation culture. It includes 11 courses with over 90 lessons built by the Confidence team at Spotify.

How long does the bootcamp take to complete?

The full bootcamp takes approximately 20 hours to complete across all 11 courses. Individual courses range from 30 minutes to 3 hours. You can learn at your own pace and pick the courses most relevant to your role.

Do I need prior experience with A/B testing or statistics?

No prior experience is required. The bootcamp starts with foundational courses like Intro to Experimentation and progressively covers more advanced topics like sequential testing and variance reduction. Each course clearly indicates which roles it is designed for.

Who created the Confidence Bootcamp?

The Confidence Bootcamp was created by the Confidence team at Spotify, the same team that builds the experimentation and feature flagging platform used across Spotify. The content reflects real-world experimentation practices used at one of the world's largest digital products.

Welcome to Intro to experimentation

Intro to experimentation is an asynchronous, self-paced course where you dive deeper into experimentation. While experimentation is a broad topic, this course focuses on teaching you the basics of A/B testing. Much of what you learn in the course also applies to other types of experiments, like rollouts.

In the course, you learn the key concepts of A/B testing and how to set up, run, and monitor an A/B test—interchangeably referred to as an experiment—using Confidence. The focus of the course is on A/B tests, but much of what you learn in the course also applies to rollouts and other types of experiments.

Before you start this course, you should have a basic understanding of what it means to run an experiment end-to-end. In our experience, this helps you internalize the concepts in our courses.

In Confidence

If you haven't already, go through the A/B test quickstart to get hands-on experience running an experiment in Confidence.

Note

There are quiz questions throughout the course to help you check your understanding of the material.

Lessons

This course consists of the following lessons:

Lesson 1: Why you should experiment

Learn about the benefits of experimentation and how it can help you make better decisions.

Not completed

Lesson 2: Experiment hypothesis

Learn how to plan your experiment and craft a solid product hypothesis.

Not completed

Lesson 3: Success and guardrail metrics

Learn about the roles of success and guardrail metrics in an experiment.

Not completed

Lesson 4: Success metrics

Learn how to select metrics and how to configure the sensitivity to detect effects.

Not completed

Lesson 5: Set up your experiment

Learn how to set up an experiment, and how to collaborate with your team in Confidence.

Not completed

Lesson 6: Calculation frequency

Learn about the options for when to calculate results and how to choose between them.

Not completed

Lesson 7: Target audience

Learn how to select who to target with your experiment.

Not completed

Lesson 8: Sample size

Learn how to calculate the sample size you need to detect the effects you are interested in.

Not completed

Lesson 9: Quality assurance

Learn about the steps you can take to ensure that your experiment works as intended before you launch.

Not completed

Lesson 10: Run your experiment

Learn about what to keep track of while your experiment is live.

Not completed

Lesson 11: Evaluate your experiment and make a decision

Learn how to interpret the results of your experiment and decide the next steps for the product you are iterating on.

Not completed

Lesson 12: Choosing between A/B tests and rollouts

Learn how to choose between A/B tests and rollouts and when to use both.

Not completed