Run a Power Analysis

In this section, you go through the steps to run a power analysis. You set up the request for the following example where you analyze an experiment with two metrics: a crash metric used as a guardrail that you analyze sequentially, and a consumption metric that you analyze once. To go through this tutorial you need to first create an analysis plan.

Run the Power Analysis

To run a power analysis, send a POST request to the power analysis endpoint with both the analysis plan and the power data:

curl -X POST "https://stats.confidence.dev/v1/powerAnalysis:run" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "plan": { ... },
    "data": { ... }
  }'

Data for the Power Analysis

To run the power analysis, you need to add power parameters to the analysis plan. For both the z-test and group sequential z-test, you need to include the mean and variance estimates. Pass the following data for the consumption metric together with the analysis plan:

{
  "hypotheses": [
    {
      "id": "consumption",
      "segments": [
        {
          "dimensions": {},
          "powerData": {
            "zTest": {
              "baselineMean": 0.3,
              "baselineVariance": 0.63
            }
          },
          "expected_sample_size": 1000
        }
      ]
    }
  ]
}

For the crash rate metric, include the following data:

{
  "hypotheses": [
    {
      "id": "crashes",
      "segments": [
        {
          "dimensions": {},
          "powerData": {
            "zTest": {
              "baselineMean": 2.5,
              "baselineVariance": 1.3
            }
          },
          "expected_sample_size": 1000
        }
      ]
    }
  ]
}

This completes the data needed for the power analysis. See the API reference for full request and response examples.

Group medium length

Group placeholder

Run the Power Analysis

Data for the Power Analysis

​Run the Power Analysis

​Data for the Power Analysis

Run the Power Analysis

Data for the Power Analysis