# Choosing a Sequential Testing Framework — Comparisons and Discussions

Last updated: 2026-05-12
Canonical source: https://confidence.spotify.com/blog/sequential-testing-comparison
Owner: Spotify AB
Authors: Mårten Schultzberg (Staff Data Scientist), Sebastian Ankargren (Senior Data Scientist)

> If this file and the page at the canonical URL disagree, the page is authoritative.

Sequential testing frameworks compared through simulation: group sequential tests, always-valid inference, and how your data infrastructure drives the choice.

## TL;DR

[Sequential tests](/glossary/sequential-testing) let you analyze experiments while data is still being collected without inflating [false positive rates](/glossary/false-positive-rate). But which sequential test should you use? The literature has developed quickly, and most leading [A/B testing](/glossary/ab-testing) companies have their own favorite.

This post compares different sequential testing frameworks using simulation results. Two main parameters should affect your choice: whether your data infrastructure provides data in batch or streaming, and whether you can estimate the maximum [sample size](/glossary/sample-size) upfront. Spotify uses [group sequential tests](/glossary/group-sequential-test) (GSTs) because they were originally designed for medical studies where data arrived in batches—similar to how our data infrastructure works.

**Read the full post on Spotify Engineering:** [Choosing a Sequential Testing Framework — Comparisons and Discussions](https://engineering.atspotify.com/2023/03/choosing-sequential-testing-framework-comparisons-and-discussions)