Skip to main content

Documentation Index

Fetch the complete documentation index at: https://confidence.spotify.com/llms.txt

Use this file to discover all available pages before exploring further.

Use this API to create and manage A/B tests. For an introduction to A/B testing concepts, see the A/B test workflows documentation.

​
Get an A/B Test

Retrieve a specific A/B test by its name to view its configuration and current state. 
curl -X GET "https://experiments.confidence.dev/v1/workflows/abtest/instances/dwimxjvyjsfno42agkl8" \
  -H "Authorization: Bearer $TOKEN"
Response: 
{
  "name": "workflows/abtest/instances/dwimxjvyjsfno42agkl8",
  "displayName": "Checkout Flow Experiment",
  "owner": "identities/cc9yglwi8kpgh0glddnkz",
  "state": "live"
}

​
List A/B Tests

List all A/B tests in your account. You can filter by state or any other criteria in the response. Use nextPageToken for getting the next page of results. 
curl -X GET "https://experiments.confidence.dev/v1/workflows/abtest/instances?pageSize=50&filter=state:live" \
  -H "Authorization: Bearer $TOKEN"
Response: 
{
  "abtests": [
    {
      "name": "workflows/abtest/instances/dwimxjvyjsfno42agkl8",
      "displayName": "Checkout Flow Experiment",
      "state": "live"
    },
    {
      "name": "workflows/abtest/instances/eioavdylbf9idlembel5",
      "displayName": "Pricing Page Test",
      "state": "live"
    }
  ],
  "nextPageToken": ""
}

​
Create an A/B Test

Create a new A/B test with treatments and metrics configuration. 
curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "displayName": "Checkout Flow Experiment",
    "flags": {
      "targetingKeySelector": "targeting_key"
    },
    "stats": {
      "testHorizonStrategy": "SEQUENTIAL"
    },
    "abtest": {
      "treatments": [
        {
          "variant": "flags/checkout-flag/variants/control",
          "weight": "5000"
        },
        {
          "variant": "flags/checkout-flag/variants/new-checkout",
          "weight": "5000"
        }
      ]
    },
    "metrics": {
      "assignmentTable": "assignmentTables/my-assignment-table",
      "entity": "entities/user",
      "bucket": "DAYS",
      "metrics": [
        {
          "metric": "metrics/conversion-rate",
          "metricRole": {
            "metricKind": "SUCCESS",
            "minimumDetectableEffect": 0.01
          },
          "preferredDirection": "INCREASE"
        }
      ]
    }
  }'
Response: 
{
  "name": "workflows/abtest/instances/checkout-experiment",
  "displayName": "Checkout Flow Experiment",
  "state": "draft",
  "createTime": "2025-11-01T10:00:00Z",
  "updateTime": "2025-11-01T10:00:00Z"
}

​
Create an A/B Test with Exposure Filters

You can create an A/B test with exposure filters to segment your analysis by different user behaviors or contexts. 
curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "displayName": "A/B test with exposure filter",
    "flags": {
      "targetingKeySelector": "targeting_key"
    },
    "stats": {
      "testHorizonStrategy": "SEQUENTIAL"
    },
    "abtest": {
      "treatments": [
        {
          "variant": "flags/page-feature-flag/variants/control",
          "weight": "5000"
        },
        {
          "variant": "flags/page-feature-flag/variants/variant",
          "weight": "5000"
        }
      ]
    },
    "metrics": {
      "assignmentTable": "assignmentTables/my-assignment-table",
      "entity": "entities/user",
      "bucket": "HOURS",
      "filters": [
        {
          "displayName": "home",
          "filter": {
            "criteria": {
              "page-1": {
                "attribute": {
                  "attribute": "page",
                  "eqRule": {
                    "value": {
                      "stringValue": "home"
                    }
                  }
                }
              }
            },
            "expression": {
              "and": {
                "operands": [{"ref": "page-1"}]
              }
            }
          },
          "factTable": "factTables/my-fact-table"
        }
      ],
      "metrics": [
        {
          "metric": "metrics/conversion-rate",
          "metricRole": {
            "metricKind": "SUCCESS",
            "minimumDetectableEffect": 0.01
          },
          "preferredDirection": "INCREASE"
        }
      ]
    }
  }'
Response: 
{
  "name": "workflows/abtest/instances/eioavdylbf9idlembel5",
  "displayName": "A/B test with exposure filter",
  "state": "draft",
  "createTime": "2025-11-13T10:00:00Z",
  "updateTime": "2025-11-13T10:00:00Z"
}

​
Set Targeting on an A/B Test

After creating an A/B test, call the UpdateSegment action with the desired targeting configuration. Include the updateMask field relative to segment. 
curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances/pinxh4k3ekqmr2fltek0:updateSegment" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "updateMask": "targeting",
    "segment": {
      "name": "segments/pztjlarivt3tyfie5gsb",
      "targeting": {
        "criteria": {
          "country": {
            "attribute": {
              "attributeName": "country",
              "eqRule": {
                "value": {
                  "stringValue": "SE"
                }
              }
            }
          }
        },
        "expression": {
          "and": {
            "operands": [{"ref": "country"}]
          }
        }
      }
    }
  }'

​
Update Allocation on an A/B Test

Call the UpdateSegment action with the allocation proportion to update the traffic allocation for an A/B test. 
curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances/pinxh4k3ekqmr2fltek0:updateSegment" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "updateMask": "allocation",
    "segment": {
      "name": "segments/pztjlarivt3tyfie5gsb",
      "allocation": {
        "proportion": {
          "value": "0.5"
        }
      }
    }
  }'

​
Action Methods

Action methods return immediately with an empty result and start the action in the background. Use the GET endpoint for the A/B test to monitor the progress by inspecting the pendingTransition field.

​
Launch an A/B Test

curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances/dwimxjvyjsfno42agkl8:launch" \
  -H "Authorization: Bearer $TOKEN"

​
End an A/B Test

curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances/dwimxjvyjsfno42agkl8:end" \
  -H "Authorization: Bearer $TOKEN"

​
Archive an A/B Test

curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances/dwimxjvyjsfno42agkl8:archive" \
  -H "Authorization: Bearer $TOKEN"

​
View Results for an A/B Test

To view statistical results for your A/B test, use a two-step process.

​
Step 1: Get the A/B Test

First, get the A/B test to retrieve the analysisResult resource name from the stats field. 
curl -X GET "https://experiments.confidence.dev/v1/workflows/abtest/instances/dwimxjvyjsfno42agkl8" \
  -H "Authorization: Bearer $TOKEN"
Response: 
{
  "name": "workflows/abtest/instances/dwimxjvyjsfno42agkl8",
  "displayName": "Checkout Flow Experiment",
  "stats": {
    "analysisResult": "workflows/abtest/instances/dwimxjvyjsfno42agkl8/analysisResults/abc123"
  }
}
If your A/B test uses multiple exposure filters, check the stats.analysisResults[] array instead. Each entry has an analysisResult resource name and the corresponding exposureFilterName.

​
Step 2: Get the Analysis Result

Use the analysisResult name from Step 1 to fetch the full statistical analysis from the Analysis Results API. 
curl -X GET "https://stats.confidence.dev/v2/workflows/abtest/instances/dwimxjvyjsfno42agkl8/analysisResults/abc123" \
  -H "Authorization: Bearer $TOKEN"
Response: 
{
  "name": "workflows/abtest/instances/dwimxjvyjsfno42agkl8/analysisResults/abc123",
  "exposureFilter": "",
  "annotations": [
    {
      "context": "OVERALL",
      "info": ["Analysis completed successfully"]
    }
  ],
  "results": [
    {
      "id": "metrics/conversion-rate",
      "statsSettings": {
        "method": "METHOD_Z_TEST",
        "adjustedAlpha": 0.05
      },
      "result": {
        "status": {
          "status": "METRIC_RESULT_STATUS_SIGNIFICANT_POSITIVE",
          "metricType": "METRIC_TYPE_SUCCESS"
        }
      }
    }
  ]
}