> ## Documentation Index
> Fetch the complete documentation index at: https://confidence.spotify.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Roll Out a Change

> Set up and launch a rollout to gradually release a change while monitoring its effect on key metrics.

By the end of this guide, you have a live rollout that serves a change to 5% of your users while Confidence monitors a guardrail metric.

<Tip>
  **Do it with AI.** Ask the [Confidence Agent](/docs/guides/work-with-the-confidence-agent) in the app or [in Slack](/docs/guides/use-confidence-in-slack):

  ```text theme={null}
  Create a rollout called "aa-rollout-<your-name-and-date>" on the Visitor entity
  with me as owner. Roll out the "treatment" variant of the "tutorial-feature"
  flag to everyone. Add "Page views per visitor" as a guardrail metric with
  non-desired direction Decrease and a non-inferiority margin of 0. Prepare it
  for me to launch.
  ```

  The agent creates the rollout. It launches it only after you confirm. Review the configuration in the app before you launch.
</Tip>

## Gradually Release Your Changes

A rollout is a type of experiment that lets you gradually introduce a change to users instead of releasing it
immediately to everyone. A rollout enables monitoring of the effect of the change on important metrics to make sure that
the change is working correctly when releasing it. If needed, you can roll back the change instantly without having
to do a new release.

In summary, a rollout gives you:

* Monitoring of a set of metrics that you choose to track, including business metrics, performance metrics, and user-behavior metrics
* Alerts if any of your metrics move in the wrong direction or if there's a sign that there's a problem with your setup
* Recommendations for how to proceed with the rollout

Some sections link to associated videos and documentation. Make use of these
resources as they contain important information that the guide doesn't cover.

## Before You Begin

This guide requires a feature flag and at least one metric. The guide
uses the flag `tutorial-feature` with a `treatment` variant, and the metric `Page views per visitor` for the
`Visitor` entity. If yours have different names, use your own flags, variants, metrics, and entities wherever the guide refers to
these.

<Tabs>
  <Tab title="With AI">
    ```text theme={null}
    Create a flag called "tutorial-feature" with variants "control" and
    "treatment" that don't change anything in my code. Then check that a metric
    called "Page views per visitor" exists for the Visitor entity.
    ```

    The assistant uses the Flags MCP tools `createFlag`, `addFlagVariant`, `listEntities`, and `listMetrics`. To create the metric with AI, see [Create a metric](/docs/guides/create-your-first-metric).
  </Tab>

  <Tab title="In the App">
    Follow [Add a feature flag](/docs/guides/add-a-feature-flag) to create the `tutorial-feature` flag, and
    [Create a metric](/docs/guides/create-your-first-metric) to create the `Page views per visitor` metric.
  </Tab>
</Tabs>

## Create a Rollout

<Tabs>
  <Tab title="With AI">
    MCP can't create rollouts. Ask the Confidence Agent in the app or in Slack:

    ```text theme={null}
    Create a draft rollout called "aa-rollout-<your-name-and-date>" on the
    Visitor entity with me as owner.
    ```

    The agent can also set the feature, audience, and metrics from the next sections in the same request. See the prompt in the **Do it with AI** box at the top of this page.
  </Tab>

  <Tab title="In the App">
    Open [Confidence](https://app.confidence.spotify.com) and
    select **Rollout** on the left sidebar.

    The overview page shows all draft, live, and ended rollouts that you have permission to view.

    Click **+ Create** in the upper right corner to create a new rollout.
  </Tab>
</Tabs>

## Set the Name, Entity, and Owner

You first need to give your rollout a name and assign an owner. Use a descriptive name that others understand.
You need to decide the entity that you want to roll out to. The rollout uses the entity to randomly assign treatment and to aggregate metrics.
For this exercise, use:

* **Name**: `aa-rollout-<your-name-and-date>`
* **Entity**: Visitor
* **Owner**: Select yourself

Click **Create**. You're now on the rollout configuration page.

## Choose the Feature

The feature you choose for your rollout determines what you want to roll out. The feature you
want to release must exist as a variant on a feature flag. Feature flags define a configuration for an
aspect of your app, website, or backend service. This step is where you select precisely what you
want to control in your rollout.

In this guide, you don't want to roll out a real feature or change for your users. Instead,
you want to use the `tutorial-feature` flag. This flag changes nothing in your code, but only
serves the purpose of testing the rollout functionality.

Click **Add variant** and select:

* **Feature flag**: `tutorial-feature`
* **Variant**: `treatment`

<Note>
  The variant selected here is the variant you want to roll out. Before you reach 100% with your
  rollout, not everyone receives the new variant. The users that don't receive the new feature instead
  get their variant set from other experiments, flag rules or the defaults defined by the clients
  that use the flag.
</Note>

## Define the Audience

The **Audience** section is where you define what your rollout should target.

The `treatment randomization unit` field determines what field in the evaluation
context of the flag that your rollout randomizes treatment assignment on.
Confidence pre-populates this field with the entity you selected in the creation
step. The [context schema](/docs/capabilities/flags/context-schema) maps the entity `Visitor` to
the field `visitor_id` (shown in parenthesis). This means that the rollout
randomizes on the value passed in the feature flag's evaluation context field `visitor_id`.

You decide the target audience for your rollout. For example, add
`country is Sweden` as an inclusion criterion if you want to target users in Sweden. The information
available in the evaluation context of the flag determines what you can target on. Read more on the
[Audiences page](/docs/capabilities/segments/audience#inclusion-criteria).

In this case, target everyone and leave the inclusion criteria empty.

<Note type="info">
  The evaluation context is information that you pass in when making the resolve call to Confidence to
  ask what variant to serve. This means what's available in the evaluation context depends on what you
  pass in. Confidence lists recently used attributes available in the evaluation context when you enter the
  name of the attribute.
</Note>

This video gives a quick overview of how targeting and evaluation contexts work in 2 minutes and 10 seconds.

<iframe className="w-full aspect-video rounded-xl" src="https://www.youtube.com/embed/rU82f1nRoGo?si=RvdoEFyt1OTdcOqr" title="Targeting and evaluation contexts in Confidence" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen />

## Add Guardrail Metrics

In this step you select the metrics you want to track to make sure that your rollout isn't
negatively affecting the user experience or performance in any way. Confidence calls such metrics
**guardrail metrics**.

<Note>
  Confidence uses the entity you selected in the creation step to decide which metrics you can select. Any metric based on the Visitor entity is available.
  You need to use a fact table that includes the Visitor entity if you want to create a new metric for it.
</Note>

In the metric section, click **Add metric** and select:

* **Metric**: `Page views per visitor` metric.
* **Non-desired direction**: `Decrease`. If you use a metric other than `Page views per visitor`, set this
  to the direction you don't want your metric to move in.
* **Non-inferiority margin**: 0.

You have successfully configured your rollout.

## Launch the Rollout

<Warning>
  This step launches your rollout. **Make sure that you have selected the `tutorial-feature` flag to avoid
  changing a real experience.**
</Warning>

<Tabs>
  <Tab title="With AI">
    MCP can't launch rollouts. Ask the Confidence Agent in the app or in Slack:

    ```text theme={null}
    Launch my rollout "aa-rollout-<your-name-and-date>".
    ```

    The agent asks you to confirm, then launches the rollout.
  </Tab>

  <Tab title="In the App">
    Click **Launch** in the top right corner.
  </Tab>
</Tabs>

## Scale Up the Reach

When you've launched your rollout, you can scale up the reach to what you want.

<Tabs>
  <Tab title="With AI">
    Ask the Confidence Agent in the app or in Slack:

    ```text theme={null}
    Set the reach of my rollout "aa-rollout-<your-name-and-date>" to 5%.
    ```

    The agent changes the reach of a live rollout when you ask. It can also set an automated ramp
    schedule, a series of reach steps that each last a set duration. Before the agent raises reach
    above the limit where monitoring stays on, it warns you that monitoring turns off permanently.
  </Tab>

  <Tab title="In the App">
    Use the slider to scale it up to 5%. Click **Save** for the new reach to apply.
  </Tab>
</Tabs>

You're now serving the `treatment` variant of the `tutorial-feature` flag to 5% of your users.

<Info>
  If there are other live rollouts or A/B tests that use the `tutorial-feature` flag, you may not receive
  5% of the traffic. Click **Flags** on the left sidebar and go to the `tutorial-feature` flag. In the
  **Rules** section, you see what rules exist on the flag. If other rules have higher priority than
  yours and use all traffic, you receive no traffic. If you want to receive traffic, you need
  to adjust the priority of your rule and move it up in the list. Read more about
  [order of rules](/docs/capabilities/flags/reorder-rules).
</Info>

**Congratulations, you have launched your first rollout!**

## Monitor the Rollout and Read Results

When you launch the rollout, Confidence calculates exposure for the rollout at
[repeated short intervals](/docs/capabilities/metrics/exposure) to make sure that the
rollout is working as expected, and that you are seeing some traffic. Keep the rollout running
and check back in tomorrow to see your first results.

<Tabs>
  <Tab title="With AI">
    ```text theme={null}
    Show the status and latest results of my rollout
    "aa-rollout-<your-name-and-date>". Are any guardrail metrics moving in the
    wrong direction?
    ```

    The assistant uses the Experiments MCP tools `list_experiments`, `get_experiment`, and `get_results`. You can also ask the Confidence Agent in the app or in Slack.
  </Tab>

  <Tab title="In the App">
    Hover the `Live` status on the right sidebar to see the current status of the checks run for the rollout.
    Click **End** in the upper right corner to end the rollout.
  </Tab>
</Tabs>

<Note>
  Remember to end your rollout within a couple of days to avoid wasting resources.
</Note>

## Bonus Step: Automate the Ramp-up of Your Rollout

Confidence lets you automate the ramp-up of your rollout. The rollout schema
allows you to define the reach steps that the rollout should take and at what
time points the increases in reach should happen. Read more about it in the
[rollout documentation](/docs/capabilities/experiments/rollouts).

## Related Resources

<CardGroup cols={2}>
  <Card title="Rollouts Reference" href="/docs/capabilities/experiments/rollouts">
    Deep dive into rollout configuration and automation
  </Card>

  <Card title="Launch an A/B Test" href="/docs/guides/run-an-ab-test">
    Compare variants with a controlled experiment
  </Card>

  <Card title="Configure Metrics" href="/docs/guides/create-your-first-metric">
    Set up metrics to monitor your rollout
  </Card>
</CardGroup>
