Who is the Confidence Bootcamp for?

The bootcamp is designed for anyone who wants to improve their experimentation skills. Courses are tailored for data scientists, analysts, engineers, product managers, and leaders — whether you are running your first A/B test or scaling an experimentation program across your organization.

Is the bootcamp free?

Yes, the Confidence Bootcamp is completely free. All 11 courses, 90+ lessons, and resources are available at no cost. You can start learning immediately without creating an account, though signing in lets you track your progress across devices.

The bootcamp covers the full experimentation lifecycle: A/B testing fundamentals, hypothesis formulation, interpreting experiment results, metrics design, sample size calculation, feature flags, and building an experimentation culture. It includes 11 courses with over 90 lessons built by the Confidence team at Spotify.

How long does the bootcamp take to complete?

The full bootcamp takes approximately 20 hours to complete across all 11 courses. Individual courses range from 30 minutes to 3 hours. You can learn at your own pace and pick the courses most relevant to your role.

Do I need prior experience with A/B testing or statistics?

No prior experience is required. The bootcamp starts with foundational courses like Intro to Experimentation and progressively covers more advanced topics like sequential testing and variance reduction. Each course clearly indicates which roles it is designed for.

Who created the Confidence Bootcamp?

The Confidence Bootcamp was created by the Confidence team at Spotify, the same team that builds the experimentation and feature flagging platform used across Spotify. The content reflects real-world experimentation practices used at one of the world's largest digital products.

Scientific product development with experimentation

Welcome

This course covers an introduction to evidence-based product development—using the scientific method to learn about our end users and make product decisions. The following pages cover the scientific method behind experimentation, how this is applied at Spotify, and how you can apply the scientific thinking to your product.

Note

There are quiz questions throughout the course to help you check your understanding of the material.

Lessons

This course consists of 9 lessons of which one is a case study:

Lesson 1: Why you should experiment

Learn about the benefits of experimentation and how it can help you make better decisions.

Not completed

Lesson 2: The scientific method

Learn about the scientific method and why it is useful in product development.

Not completed

Lesson 3: Randomized controlled trials

Learn about randomization of the treatment assignment and the role it plays in experimentation.

Not completed

Lesson 4: Experiment hypothesis

Learn how to specify a precise and testable hypothesis for your experiment.

Not completed

Lesson 5: Case study — shuffle button in Spotify

Practice hypothesis creation on a real Spotify experiment.

Not completed

Lesson 6: Why do we need statistics

Learn how statistics helps you quantify the uncertainty and make risk-informed decisions.

Not completed

Lesson 7: Success metrics

Learn how to select metrics and how to configure the sensitivity to detect effects.

Not completed

Lesson 8: Detectable effects and sample size

Learn how to set the sensitivity of your experiment using the minimum detectable effect and how that affects the sample size requirements.

Not completed

Lesson 9: Make a decision

Learn how to make decisions in a scientifically sound way.

Not completed