> ## Documentation Index
> Fetch the complete documentation index at: https://confidence.spotify.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Analysis Quickstart

> Analyze a past experiment administered through another feature flagging provider in Confidence with the help of this quickstart.

export const NotForSpotify = () => <Note>
    <strong>This page doesn't apply at Spotify.</strong> Use the Spotify
    documentation in the sidebar.
  </Note>;

{user?.groups?.includes("spotify") && <NotForSpotify />}

<Tip>
  Don't analyze live experiments with an analysis in
  Confidence. This experiment type specializes in analyzing past experiments and is
  not suitable for tracking a live experiment.
</Tip>

## Before You Begin

To run an analysis, you need:

* an [entity](/docs/metrics/entities)
* a [fact table](/docs/metrics/fact-tables)
* a [metric](../metrics)

Use the [metrics quickstart](/docs/quickstarts/configure-metric) to create an entity, fact table, and a metric.

## Step 1: Create the Assignment Table

An assignment table is a table that has data about assignments of entities to variants in experiments.
For your analysis to be able to read the data, you need to create an assignment table in Confidence.
Follow the steps on the [assignment table page](/docs/metrics/assignment-tables) to
create your table.

### Assignment Table from Optimizely Decision Events

You can analyze past or current experiments in Optimizely with Confidence.
To do so, you need to [export the decision events](https://docs.developers.optimizely.com/experimentation-data/docs/enriched-events-export)
from Optimizely to a table in your data warehouse.
A decision event is an event that Optimizely records when a visitor is exposed to an experiment.
Decision events in Optimizely correspond to assignments in Confidence.

The information Confidence requires is available in the columns:

* `experiment_id`: column with identifiers of the experiments
* `variation_id`: column with identifiers of the variants
* `visitor_id`: column with identifiers of the entities in the experiments, like users and visitors
* `timestamp`: column with timestamps of the events

To set up the assignment table in Confidence, follow these steps:

1. [Export](https://docs.developers.optimizely.com/experimentation-data/docs/enriched-events-export)
   the [decision events](https://docs.developers.optimizely.com/experimentation-data/docs/enriched-events-data-specification#decisions-1)
   from Optimizely to a table in your data warehouse. If you want to analyze a running experiment,
   you need to schedule the export to happen at a regular cadence.
2. If you don't have one already, [create an entity](/docs/metrics/entities) in Confidence that identifies the
   entity that's recorded in the `visitor_id` column of the decision events table.
3. [Create an assignment table](/docs/metrics/assignment-tables) in Confidence that points to the decision events table.
   * Set the exposure key column to `experiment_id`.
   * Set the variant key column to `variation_id`.
   * Set the entity to the entity you created in step 2.
   * Set the entity column to `visitor_id`.
   * Set the timestamp column to `timestamp`.

The columns `experiment_id` and `variation_id` must be strings to be selectable
as exposure key and variant. The type of the `visitor_id` must match the primary key type of the
entity you created in step 2, such as a string. The `timestamp` column must be a timestamp.

## Step 2: Create an Analysis

Open [Confidence](https://app.confidence.spotify.com) and
select **Analyses** on the left sidebar.
The overview page shows all draft, live, and ended analyses that you have permission to view.

Click **+ Create** in the upper right corner to create a new analysis.

## Step 3: Name, Assignment table, Entity and Owner

Specify which entity the experiment used for randomizing the treatment assignment, for example `User`.
Confidence uses the entity to map the metrics to the units in the analysis.

You first need to give your analysis a name and assign an owner. Use a descriptive name that others understand.
For this exercise, use:

* **Name**: `analysis-<your-name-and-date>`
* **Assignment table**: Select which assignment table to use for this analysis. This option only appears if you have more than one assignment table in Confidence.
* **Entity**: Select the entity that represents the unit you experiment on.
* **Owner**: Select yourself

Click **Create**. You're now on the analysis design page.

## Step 4: Dates and Exposure Key

Select the date range for the experiment to avoid scanning unnecessary data.

The analysis start date is the first day of the experiment that you want to analyze.
If the experiment started on 2023-01-01, input 2023-01-01 as the start date to begin
analyzing it on the day it started.

Enter the identifier of the experiment in the assignment data as the exposure key. For example,
if your experiment identifier is `experiment-123`, enter `experiment-123` as the exposure key. If
you're analyzing an experiment based on decision events exported from Optimizely, your exposure key
is the identifier for your experiment that's available in the `experiment_id` column in your decision events table.

## Step 5: Treatments

To configure the treatment groups for your analysis, you need to enter the variant keys for the treatment groups.
Confidence pre-populates the treatment variant list with the unique values found
in the variant key column of the assignment table for the selected exposure key.
The variant key is the identifier for each group in the experiment that, together with the exposure key, uniquely
identifies the relevant group in the assignment table. For example, if the control group is `default-style`, enter `default-style` as the
variant key.

To set up your treatments:

<Steps>
  <Step title="Click Add control in the Treatments section" />

  <Step title="Enter the control group identifier">
    Enter the identifier of the control group in the **Variant key** field. Optionally enter a display name in the **Name** field and upload an image of the variant. Click **Save and add another**.
  </Step>

  <Step title="Repeat the process for each treatment group" />

  <Step title="Adjust the weights of the treatments">
    Adjust the weights of the treatments to match the weights you used in the underlying randomization.
  </Step>
</Steps>

<Tip>
  While the provided treatment split doesn't affect traffic, it
  is important that you specify the intended treatment split from the original
  experiment. Confidence uses the given split to validate that the randomization is
  correct, a central validity check in an experiment. If the observed proportions
  of the variants don't match the pre-specified split, the analysis triggers the check for
  balanced traffic.
</Tip>

<Note>
  If you're analyzing an experiment based on decision events exported from Optimizely, your variant key
  is the identifier for the treatment group that's available in the `variation_id` column in your decision events table.
</Note>

## Step 6: Metrics

To measure the outcome of the experiment, you need to add metrics to the analysis. In Confidence,
metrics are either **Success metrics** or **Guardrail metrics**. Add your metric as a success metric
if you hope to see an improvement in the metric. For example, with your change you hope to see an increase
in the number of purchases per user. Add your metric as a guardrail metric if you don't expect to see a change,
but you want to make sure the change doesn't have a negative impact. For example, with your change you don't want
to see an increase in the number of returned items per user.

To add your metrics to the analysis:

<Steps>
  <Step title="Click Add metric in either the Success metrics or Guardrail metrics section" />

  <Step title="Select and configure the metric">
    Select the metric you want to add to the analysis. For success metrics, select the **Preferred direction** and enter an **MDE** (minimum detectable effect). The MDE represents the size of the change you're interested in finding. For guardrail metrics, select the **Preferred direction** and enter the **NIM** (non-inferiority margin). The NIM represents your tolerance for a negative change. Click **Add metrics**.
  </Step>
</Steps>

Read more about [minimum detectable effects and non-inferiority margins](/docs/experiments/design/effect-sizes).

Your analysis can have required metrics added from the surface that the analysis belongs to. Read more about [required metrics](../surfaces#required-metrics).

## Step 7: Calculate

Review your setup and click **Calculate** to run the analysis. You are now on the Result page.

You can add exploratory analyses on the Result page. Click **Explore** on the Metrics result section.

<Note>
  You can go back and change settings on the Design tab. If you change
  certain settings, like exposure key or treatments, you need to recalculate the
  analysis to get back to the Result tab.
</Note>

## Related Resources

<CardGroup cols={2}>
  <Card title="Analysis" href="/docs/experiments/workflows/analysis">
    Deep dive into analysis configuration options
  </Card>

  <Card title="Assignment Tables" href="/docs/metrics/assignment-tables">
    Understand how to configure assignment data sources
  </Card>

  <Card title="Launch an A/B Test" href="/docs/quickstarts/launch-abtest">
    Run new experiments directly in Confidence
  </Card>

  <Card title="Migrate to Confidence" href="/docs/migrations/overview">
    Move your flags and code from PostHog, Eppo, Statsig, or Optimizely
  </Card>
</CardGroup>