Who is the Confidence Bootcamp for?

The bootcamp is designed for anyone who wants to improve their experimentation skills. Courses are tailored for data scientists, analysts, engineers, product managers, and leaders — whether you are running your first A/B test or scaling an experimentation program across your organization.

Is the bootcamp free?

Yes, the Confidence Bootcamp is completely free. All 11 courses, 90+ lessons, and resources are available at no cost. You can start learning immediately without creating an account, though signing in lets you track your progress across devices.

The bootcamp covers the full experimentation lifecycle: A/B testing fundamentals, hypothesis formulation, interpreting experiment results, metrics design, sample size calculation, feature flags, and building an experimentation culture. It includes 11 courses with over 90 lessons built by the Confidence team at Spotify.

How long does the bootcamp take to complete?

The full bootcamp takes approximately 20 hours to complete across all 11 courses. Individual courses range from 30 minutes to 3 hours. You can learn at your own pace and pick the courses most relevant to your role.

Do I need prior experience with A/B testing or statistics?

No prior experience is required. The bootcamp starts with foundational courses like Intro to Experimentation and progressively covers more advanced topics like sequential testing and variance reduction. Each course clearly indicates which roles it is designed for.

Who created the Confidence Bootcamp?

The Confidence Bootcamp was created by the Confidence team at Spotify, the same team that builds the experimentation and feature flagging platform used across Spotify. The content reflects real-world experimentation practices used at one of the world's largest digital products.

Course wrap up

Congratulations! You have finished Interpreting experiment results!

You can now open any experiment results page in Confidence and know exactly what you are looking at. To recap what you have covered:

The results page has three sections: Spotlight, Health checks, and Metrics, each answering a different question.
The control variant and treatment variant means are averages of what actually happened for real users, and effects are always shown as relative % changes to make them comparable across metrics.
A confidence interval tells you both where the effect likely is and how precisely you have measured it. A wide CI means you need more data, not that there is no effect.
Status labels differ between success and guardrail metrics because they answer different questions: "did it improve?" versus "did it break anything?"
The SRM check is the most critical health check. If it fails, no metric result can be trusted.
Variance reduction makes estimates more precise by using pre-experiment behavior to remove noise. The numbers look slightly adjusted, but you interpret them the same way.
Your choice of evaluation strategy determines when results are valid to act on. Deterioration checks always run sequentially regardless of that choice.
The Spotlight synthesizes everything (health, success metrics, and guardrail metrics) into one recommendation per treatment variant.
Explorations are for learning and hypothesis generation, not for deciding whether an experiment succeeded.

What to explore next

If you want to go deeper on the statistical foundations behind what you learned here, the A primer on hypothesis testing course covers the mechanics of how hypothesis tests work and where p-values and significance thresholds come from.

To learn about more advanced experiment configurations, including guardrail metrics with non-inferiority margins and how to choose between sequential and non-sequential tests, check out Advance your experimentation.

Go back to my learning page to keep learning!

Course wrap up

Congratulations! You have finished Interpreting experiment results!

You can now open any experiment results page in Confidence and know exactly what you are looking at. To recap what you have covered:

The results page has three sections: Spotlight, Health checks, and Metrics, each answering a different question.
The control variant and treatment variant means are averages of what actually happened for real users, and effects are always shown as relative % changes to make them comparable across metrics.
A confidence interval tells you both where the effect likely is and how precisely you have measured it. A wide CI means you need more data, not that there is no effect.
Status labels differ between success and guardrail metrics because they answer different questions: "did it improve?" versus "did it break anything?"
The SRM check is the most critical health check. If it fails, no metric result can be trusted.
Variance reduction makes estimates more precise by using pre-experiment behavior to remove noise. The numbers look slightly adjusted, but you interpret them the same way.
Your choice of evaluation strategy determines when results are valid to act on. Deterioration checks always run sequentially regardless of that choice.
The Spotlight synthesizes everything (health, success metrics, and guardrail metrics) into one recommendation per treatment variant.
Explorations are for learning and hypothesis generation, not for deciding whether an experiment succeeded.

What to explore next

Go back to my learning page to keep learning!