When Proxy Metrics Break: How Optimizing for Proxies Can Backfire
Learn how wrong things can go when proxy metrics start to influence product development, and how to use them safely.
Read more
Spotify R&D
Why We Use Separate Tech Stacks for Personalization and Experimentation
Why we maintain distinct technology stacks for personalization and experimentation.
Read more
Two Questions Every Experiment Should Answer
Learn how to never run an experiment without learning something.
Read more
The Feature Flag Toolbox: Cloud, Edge, and Local
Learn about how Confidence enables feature flagging for all needs.
Read more
How Experimental Evidence Travels Through Your Organization: Why Better May Be Worse
Learn about how experimental evidence travels through your organization, and why better may be worse.
Read more
Spotify R&D
Beyond Winning: Spotify's Experiments with Learning Framework
How the Experiments with Learning (EwL) framework measures experimentation success beyond win rates.
Read more
A/B Test Bandwidth: The Currency of Innovation
Learn about how experiment bandwidth is the currency of innovation, and how you can improve yours.
Read more
Experiments with Smaller Samples
Learn about how you can benefit from experimentation even when your samples are smaller than you wish.
Read more
Reduce Dilution and Improve Sensitivity with Trigger Analysis
Read about how trigger analysis can help improve sensitivity by narrowing the exposure definition.
Read more
Spotify R&D
Fixed-Power Designs: It's Not IF You Peek, It's WHAT You Peek at
A new experimental design that lets you estimate sample size during the experiment without compromising inference.
Read more
Better Product Decisions with Guardrail Metrics
Read about how to improve the quality of product decisions by using guardrail metrics in experiments.
Read more
Collaboration Fuels Efficient Experimentation
Read more about how Confidence makes your collaboration thrive.
Read more
Spotify R&D
Risk-Aware Product Decisions in A/B Tests with Multiple Metrics
A framework for combining multiple metrics in A/B tests into a single product decision.
Read more
Experiment like Spotify: Analysis of Experiments
Read more about how you can use Confidence to analyze experiments like Spotify.
Read more
Experiment like Spotify: Feature Flags
Read more about feature flags and the functionality Confidence offers.
Read more
Experiment like Spotify: A/B Tests and Rollouts
Read more about the difference between an A/B test and a rollout, and how they're used at Spotify.
Read more
Experiment like Spotify: With Confidence
Learn more about Confidence and what it can do for you.
Read more
Spotify R&D
The Peeking Problem 2.0 (Part 2): Sequential Testing
Solutions for handling sequential testing with longitudinal data.
Read more
Spotify R&D
The Peeking Problem 2.0 (Part 1)
A new challenge in sequential testing with longitudinal data that can inflate false positive rates.
Read more
Spotify R&D
Choosing a Sequential Testing Framework — Comparisons and Discussions
Comparing different sequential testing frameworks for online experiments.
Read more
Spotify R&D
Search Journey Towards Better Experimentation Practices
How the Search team built experimentation practices from the ground up.
Read more