Manage A/B Tests - Confidence Documentation

Use this API to create and manage A/B tests. For an introduction to A/B testing concepts, see the A/B test workflows documentation. All requests require a bearer token in the Authorization header. See Get Started with Confidence APIs for how to obtain one using your client ID and secret.

Get an A/B Test

Retrieve a specific A/B test by its name to view its configuration and current state.

curl -X GET "https://experiments.confidence.dev/v1/workflows/abtest/instances/dwimxjvyjsfno42agkl8" \
  -H "Authorization: Bearer $TOKEN"

Response:

{
  "name": "workflows/abtest/instances/dwimxjvyjsfno42agkl8",
  "displayName": "Checkout Flow Experiment",
  "owner": "identities/cc9yglwi8kpgh0glddnkz",
  "state": "live"
}

List A/B Tests

List all A/B tests in your account. You can filter by state or any other criteria in the response. Use nextPageToken for getting the next page of results.

curl -X GET "https://experiments.confidence.dev/v1/workflows/abtest/instances?pageSize=50&filter=state:live" \
  -H "Authorization: Bearer $TOKEN"

Response:

{
  "abtests": [
    {
      "name": "workflows/abtest/instances/dwimxjvyjsfno42agkl8",
      "displayName": "Checkout Flow Experiment",
      "state": "live"
    },
    {
      "name": "workflows/abtest/instances/eioavdylbf9idlembel5",
      "displayName": "Pricing Page Test",
      "state": "live"
    }
  ],
  "nextPageToken": ""
}

Create an A/B Test

Create a new A/B test with treatments and metrics configuration.

curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "displayName": "Checkout Flow Experiment",
    "flags": {
      "targetingKeySelector": "targeting_key"
    },
    "stats": {
      "testHorizonStrategy": "SEQUENTIAL"
    },
    "abtest": {
      "treatments": [
        {
          "variant": "flags/checkout-flag/variants/control",
          "weight": "5000"
        },
        {
          "variant": "flags/checkout-flag/variants/new-checkout",
          "weight": "5000"
        }
      ]
    },
    "metrics": {
      "assignmentTable": "assignmentTables/my-assignment-table",
      "entity": "entities/user",
      "bucket": "DAYS",
      "metrics": [
        {
          "metric": "metrics/conversion-rate",
          "metricRole": {
            "metricKind": "SUCCESS",
            "minimumDetectableEffect": 0.01
          },
          "preferredDirection": "INCREASE"
        }
      ]
    }
  }'

Response:

{
  "name": "workflows/abtest/instances/checkout-experiment",
  "displayName": "Checkout Flow Experiment",
  "state": "draft",
  "createTime": "2025-11-01T10:00:00Z",
  "updateTime": "2025-11-01T10:00:00Z"
}

Create an A/B Test with Exposure Filters

You can create an A/B test with exposure filters to segment your analysis by different user behaviors or contexts.

curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "displayName": "A/B test with exposure filter",
    "flags": {
      "targetingKeySelector": "targeting_key"
    },
    "stats": {
      "testHorizonStrategy": "SEQUENTIAL"
    },
    "abtest": {
      "treatments": [
        {
          "variant": "flags/page-feature-flag/variants/control",
          "weight": "5000"
        },
        {
          "variant": "flags/page-feature-flag/variants/variant",
          "weight": "5000"
        }
      ]
    },
    "metrics": {
      "assignmentTable": "assignmentTables/my-assignment-table",
      "entity": "entities/user",
      "bucket": "HOURS",
      "filters": [
        {
          "displayName": "home",
          "filter": {
            "criteria": {
              "page-1": {
                "attribute": {
                  "attribute": "page",
                  "eqRule": {
                    "value": {
                      "stringValue": "home"
                    }
                  }
                }
              }
            },
            "expression": {
              "and": {
                "operands": [{"ref": "page-1"}]
              }
            }
          },
          "factTable": "factTables/my-fact-table"
        }
      ],
      "metrics": [
        {
          "metric": "metrics/conversion-rate",
          "metricRole": {
            "metricKind": "SUCCESS",
            "minimumDetectableEffect": 0.01
          },
          "preferredDirection": "INCREASE"
        }
      ]
    }
  }'

Response:

{
  "name": "workflows/abtest/instances/eioavdylbf9idlembel5",
  "displayName": "A/B test with exposure filter",
  "state": "draft",
  "createTime": "2025-11-13T10:00:00Z",
  "updateTime": "2025-11-13T10:00:00Z"
}

Set Targeting on an A/B Test

After creating an A/B test, call the UpdateSegment action with the desired targeting configuration. Include the updateMask field relative to segment.

curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances/pinxh4k3ekqmr2fltek0:updateSegment" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "parameters": {
      "updateMask": "targeting",
      "segment": {
        "name": "segments/pztjlarivt3tyfie5gsb",
        "targeting": {
          "criteria": {
            "country": {
              "attribute": {
                "attributeName": "country",
                "eqRule": {
                  "value": {
                    "stringValue": "SE"
                  }
                }
              }
            }
          },
          "expression": {
            "and": {
              "operands": [{"ref": "country"}]
            }
          }
        }
      }
    }
  }'

Update Allocation on an A/B Test

Call the UpdateSegment action with the allocation proportion to update the traffic allocation for an A/B test.

curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances/pinxh4k3ekqmr2fltek0:updateSegment" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "parameters": {
      "updateMask": "allocation",
      "segment": {
        "name": "segments/pztjlarivt3tyfie5gsb",
        "allocation": {
          "proportion": {
            "value": "0.5"
          }
        }
      }
    }
  }'

Action Methods

Action methods return immediately with an empty result and start the action in the background. Use the GET endpoint for the A/B test to monitor the progress by inspecting the pendingTransition field.

Launch an A/B Test

curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances/dwimxjvyjsfno42agkl8:launch" \
  -H "Authorization: Bearer $TOKEN"

End an A/B Test

curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances/dwimxjvyjsfno42agkl8:end" \
  -H "Authorization: Bearer $TOKEN"

Archive an A/B Test

curl -X POST "https://experiments.confidence.dev/v1/workflows/abtest/instances/dwimxjvyjsfno42agkl8:archive" \
  -H "Authorization: Bearer $TOKEN"

View Results for an A/B Test

To view statistical results for your A/B test, use a two-step process.

Step 1: Get the A/B Test

First, get the A/B test to retrieve the analysisResult resource name from the stats field.

curl -X GET "https://experiments.confidence.dev/v1/workflows/abtest/instances/dwimxjvyjsfno42agkl8" \
  -H "Authorization: Bearer $TOKEN"

Response:

{
  "name": "workflows/abtest/instances/dwimxjvyjsfno42agkl8",
  "displayName": "Checkout Flow Experiment",
  "stats": {
    "analysisResult": "workflows/abtest/instances/dwimxjvyjsfno42agkl8/analysisResults/abc123"
  }
}

If your A/B test uses multiple exposure filters, check the stats.analysisResults[] array instead. Each entry has an analysisResult resource name and the corresponding exposureFilterName.

Step 2: Get the Analysis Result

Use the analysisResult name from Step 1 to fetch the full statistical analysis from the Analysis Results API.

curl -X GET "https://stats.confidence.dev/v2/workflows/abtest/instances/dwimxjvyjsfno42agkl8/analysisResults/abc123" \
  -H "Authorization: Bearer $TOKEN"

Response:

{
  "name": "workflows/abtest/instances/dwimxjvyjsfno42agkl8/analysisResults/abc123",
  "exposureFilter": "",
  "annotations": [
    {
      "context": "OVERALL",
      "info": ["Analysis completed successfully"]
    }
  ],
  "results": [
    {
      "id": "metrics/conversion-rate",
      "statsSettings": {
        "method": "METHOD_Z_TEST",
        "adjustedAlpha": 0.05
      },
      "result": {
        "status": {
          "status": "METRIC_RESULT_STATUS_SIGNIFICANT_POSITIVE",
          "metricType": "METRIC_TYPE_SUCCESS"
        }
      }
    }
  ]
}

​Get an A/B Test

​List A/B Tests

​Create an A/B Test

​Create an A/B Test with Exposure Filters

​Set Targeting on an A/B Test

​Update Allocation on an A/B Test

​Action Methods

​Launch an A/B Test

​End an A/B Test

​Archive an A/B Test

​View Results for an A/B Test

​Step 1: Get the A/B Test

​Step 2: Get the Analysis Result

Get an A/B Test

List A/B Tests

Create an A/B Test

Create an A/B Test with Exposure Filters

Set Targeting on an A/B Test

Update Allocation on an A/B Test

Action Methods

Launch an A/B Test

End an A/B Test

Archive an A/B Test

View Results for an A/B Test

Step 1: Get the A/B Test

Step 2: Get the Analysis Result