Skip to main content

Documentation Index

Fetch the complete documentation index at: https://confidence.spotify.com/llms.txt

Use this file to discover all available pages before exploring further.

In this section, you go through the steps to run a power analysis. You set up the request for the following example where you analyze an experiment with two metrics: a crash metric used as a guardrail that you analyze sequentially, and a consumption metric that you analyze once. To go through this tutorial you need to first create an analysis plan.

​
Run the Power Analysis

To run a power analysis, send a POST request to the power analysis endpoint with both the analysis plan and the power data: 
curl -X POST "https://stats.confidence.dev/v1/powerAnalysis:run" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "plan": { ... },
    "data": { ... }
  }'

​
Data for the Power Analysis

To run the power analysis, you need to add power parameters to the analysis plan. For both the z-test and group sequential z-test, you need to include the mean and variance estimates. Pass the following data for the consumption metric together with the analysis plan: 
{
  "hypotheses": [
    {
      "id": "consumption",
      "segments": [
        {
          "dimensions": {},
          "powerData": {
            "zTest": {
              "baselineMean": 0.3,
              "baselineVariance": 0.63
            }
          },
          "expected_sample_size": 1000
        }
      ]
    }
  ]
}
For the crash rate metric, include the following data: 
{
  "hypotheses": [
    {
      "id": "crashes",
      "segments": [
        {
          "dimensions": {},
          "powerData": {
            "zTest": {
              "baselineMean": 2.5,
              "baselineVariance": 1.3
            }
          },
          "expected_sample_size": 1000
        }
      ]
    }
  ]
}
This completes the data needed for the power analysis. See the API reference for full request and response examples.