> ## Documentation Index
> Fetch the complete documentation index at: https://confidence.spotify.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Configure Sequential Testing

> Use sequential testing methods in the Stats API to analyze experiments during data collection.

export const HowToSchema = ({name, description, steps = []}) => {
  const schema = {
    "@context": "https://schema.org",
    "@type": "HowTo",
    name,
    description
  };
  if (steps.length > 0) {
    schema.step = steps.map((s, i) => ({
      "@type": "HowToStep",
      position: i + 1,
      name: typeof s === "string" ? s : s.name,
      text: typeof s === "string" ? s : s.text || s.name
    }));
  }
  return <script type="application/ld+json" dangerouslySetInnerHTML={{
    __html: JSON.stringify(schema)
  }} />;
};

<HowToSchema
  name="Configure Sequential Testing"
  description="Use sequential testing methods in the Stats API to analyze experiments during data collection."
  steps={[
{
  name: "Fixed Horizon vs Sequential Tests",
  text: "The Stats API supports two testing approaches: | Method | Description | When to Use | |--------|-------------|-------------|",
},
{
  name: "Configure a Group Sequential Test",
  text: "Group sequential tests provide valid statistical conclusions even when you analyze results multiple times during data collection. You need to specify the expected maximum sample size upfront. In your analysis plan, specify gstZTest in the hypothesis segments: The maxSampleSize parameter helps the test allocate the false positive rate optimally across analyses. Your estimate doesn't need to be exact, but a reasonable estimate improves power.",
},
{
  name: "Configure a Fixed Horizon Test",
  text: "For metrics you only analyze once at the end of the experiment, use the standard zTest:",
},
{
  name: "Provide Time-Series Data",
  text: "Sequential tests require time-series data to track how results evolve. Structure your data with timeLabel values containing cumulative statistics up to each time point:",
},
]}
/>

Sequential tests allow you to analyze experiment results during data collection without invalidating statistical conclusions. The Stats API uses the group sequential test method (`gstZTest`) for sequential analysis.

## Fixed Horizon vs Sequential Tests

The Stats API supports two testing approaches:

| Method     | Description             | When to Use                                                          |
| ---------- | ----------------------- | -------------------------------------------------------------------- |
| `zTest`    | Fixed horizon z-test    | When you analyze results once at the end of the experiment           |
| `gstZTest` | Group sequential z-test | When you want to analyze results continuously during data collection |

## Configure a Group Sequential Test

Group sequential tests provide valid statistical conclusions even when you analyze results multiple times during data collection. You need to specify the expected maximum sample size upfront.

In your analysis plan, specify `gstZTest` in the hypothesis segments:

```json theme={null}
{
  "hypotheses": [
    {
      "id": "my-metric",
      "type": {
        "superiority": {
          "preferredDirection": "INCREASE",
          "minimumDetectableEffect": 0.03
        }
      },
      "segments": [
        {
          "dimensions": {},
          "params": {
            "gstZTest": {
              "maxSampleSize": 10000
            }
          }
        }
      ]
    }
  ]
}
```

The `maxSampleSize` parameter helps the test allocate the false positive rate optimally across analyses. Your estimate doesn't need to be exact, but a reasonable estimate improves power.

## Configure a Fixed Horizon Test

For metrics you only analyze once at the end of the experiment, use the standard `zTest`:

```json theme={null}
{
  "hypotheses": [
    {
      "id": "my-metric",
      "type": {
        "superiority": {
          "preferredDirection": "INCREASE",
          "minimumDetectableEffect": 0.03
        }
      },
      "segments": [
        {
          "dimensions": {},
          "params": {
            "zTest": {}
          }
        }
      ]
    }
  ]
}
```

## Provide Time-Series Data

Sequential tests require time-series data to track how results evolve. Structure your data with `timeLabel` values containing cumulative statistics up to each time point:

```json theme={null}
{
  "id": "my-metric",
  "segments": [
    {
      "dimensions": {},
      "groups": [
        {
          "group": "control",
          "data": {
            "gstZTest": {
              "summary": {
                "data": [
                  {
                    "timeLabel": "2024-01-01",
                    "value": {
                      "mean": 2.0,
                      "variance": 2.28,
                      "count": 100
                    }
                  },
                  {
                    "timeLabel": "2024-01-02",
                    "value": {
                      "mean": 1.99,
                      "variance": 2.29,
                      "count": 200
                    }
                  }
                ]
              }
            }
          }
        }
      ]
    }
  ]
}
```

## Related Resources

<CardGroup cols={2}>
  <Card title="Sequential Tests" href="/docs/experiments/stats/sequential-tests">
    Learn about sequential testing concepts
  </Card>

  <Card title="Run an Analysis" href="/docs/api/how-to-guides/stats/run-analysis">
    Complete analysis tutorial
  </Card>

  <Card title="Create Analysis Plan" href="/docs/api/how-to-guides/stats/create-analysis-plan">
    Set up your analysis plan
  </Card>

  <Card title="Statistical Tests" href="/docs/experiments/stats/stat-tests">
    Understand test types
  </Card>
</CardGroup>