Scientific product development with experimentation
Welcome
This course covers an introduction to evidence-based product development—using the scientific method to learn about our end users and make product decisions. The following pages cover the scientific method behind experimentation, how this is applied at Spotify, and how you can apply the scientific thinking to your product.
There are quiz questions throughout the course to help you check your understanding of the material.
Lessons
This course consists of 9 lessons of which one is a case study:
Lesson 1: Why you should experiment
Learn about the benefits of experimentation and how it can help you make better decisions.
Lesson 2: The scientific method
Learn about the scientific method and why it is useful in product development.
Lesson 3: Randomized controlled trials
Learn about randomization of the treatment assignment and the role it plays in experimentation.
Lesson 4: Experiment hypothesis
Learn how to specify a precise and testable hypothesis for your experiment.
Lesson 5: Case study — shuffle button in Spotify
Practice hypothesis creation on a real Spotify experiment.
Lesson 6: Why do we need statistics
Learn how statistics helps you quantify the uncertainty and make risk-informed decisions.
Lesson 7: Success metrics
Learn how to select metrics and how to configure the sensitivity to detect effects.
Lesson 8: Detectable effects and sample size
Learn how to set the sensitivity of your experiment using the minimum detectable effect and how that affects the sample size requirements.
Lesson 9: Make a decision
Learn how to make decisions in a scientifically sound way.