> ## Documentation Index
> Fetch the complete documentation index at: https://confidence.spotify.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Metrics Data Processing

> Understand how Confidence processes your metrics data through entities, fact tables, assignment tables, and exposure calculations.

## Introduction

Confidence processes metrics data through four core components that work together to calculate experiment results:

* **Entities** - The units you're measuring (users, orders, sessions)
* **Fact Tables** - Where your measurement data lives in the data warehouse
* **Assignment Tables** - Where you record variant assignments
* **Exposure** - When measurement starts for each entity

Understanding how these components interact is essential for designing effective experiments and creating accurate metrics.

## The Four Components

### Entities

An entity is the "thing" you're counting or measuring in your experiments. Entities represent the units that can be uniquely identified, measured, and randomized.

**Key characteristic**: Each entity has a unique identifier with a specific data type.

**Common examples**:

* **User** - Identified by `user_id` (string UUID or integer)
* **Order** - Identified by `order_id`
* **Session** - Identified by `session_id`

Entities are the foundation of your metrics system. Both your fact tables and assignment tables reference entities through their unique identifiers.

### Fact Tables

Fact tables contain the events and measurements you want to analyze. These tables live in the data warehouse and describe what your entities are doing.

**Required structure**:

```sql theme={null}
SELECT
  user_id,              -- Entity identifier
  event_timestamp,      -- When the fact occurred
  revenue_amount,       -- Measurement (optional)
  platform              -- Dimension (optional)
FROM your_warehouse.events.purchases
WHERE event_timestamp BETWEEN {START_TIME} AND {END_TIME}
```

<Note>
  `{START_TIME}` and `{END_TIME}` are placeholder variables that Confidence automatically replaces with the actual time range when executing queries. These vary depending on the metric's requirements and ensure that only relevant data within the experiment's time window is queried.
</Note>

**Required columns**:

* `entity_id` - The unique identifier linking to an entity (for example, `user_id`, `order_id`)
* `timestamp` - When the event occurred

**Optional columns**:

* **Measurements** - Numeric or boolean values you want to aggregate (for example, `revenue_amount`, `duration`, `is_successful`)
* **Dimensions** - Categorical values for segmentation (for example, `platform`, `device_type`)

**Best practices**:

* Always index the timestamp column (time partitioning in BigQuery) to avoid full table scans
* Index entity columns (clustering in BigQuery) to improve join performance
* Remove duplicate rows to avoid double counting
* Validate the distribution of nulls and zeros in your data

### Assignment Tables

Assignment tables record which variant each entity was assigned to during an experiment. This data typically comes from your feature flagging system.

**Structure**:

| assignment\_time | user\_id | flag | rule | variant | assignment\_id |
| - | - | - | - | - | - |
| 2023-05-20 00:04:25 | user5125 | flags/test-flag | flags/test-flag/rules/rule123 | flags/test-flag/variants/control | control |
| 2023-05-20 05:01:21 | user7231 | flags/test-flag | flags/test-flag/rules/rule123 | flags/test-flag/variants/treatment | treatment |
| 2023-05-20 23:13:53 | user3512 | flags/other-test-flag | flags/other-test-flag/rules/rule123 | flags/other-test-flag/variants/treatment | control |

**Required columns**:

* `timestamp` - When the assignment occurred (for example, `assignment_time`)
* `entity_id` - The identifier of the entity that received the assignment (for example, `user_id`)
* `exposure_key` - Identifies which experiment the assignment belongs to (for example, `rule`)
* `variant_key` - Identifies which variant was assigned (for example, `assignment_id`)

**Example SQL**:

```sql theme={null}
SELECT
  assignment_time,
  user_id,
  rule AS exposure_key,
  assignment_id AS variant_key
FROM your_warehouse.confidence.assignments
WHERE assignment_time BETWEEN {START_TIME} AND {END_TIME}
```

### Exposure

Exposure defines when to start measuring an entity in an experiment. Confidence calculates exposure from assignment data, representing the **first time** an entity was assigned to a variant.

**Why it matters**:

* Ensures fair comparison between variants by measuring from a consistent starting point
* Prevents bias from entities that were assigned at different times
* Defines the time window for including facts in metric calculations

**How Confidence calculates exposure**:

1. Queries the assignment table for a specific experiment (filtered by exposure key)
2. For each entity, finds the **first assignment** timestamp
3. Groups by variant to create separate exposure cohorts
4. Writes the exposure table to the data warehouse for reuse

**Exposure example**:

If user `user5125` was assigned to the control variant at `2023-05-20 00:04:25`, their exposure time is `2023-05-20 00:04:25`. Confidence includes all facts for this user after this timestamp (within the metric's time window) in the metric calculation.

## How They Work Together

The four components form a data processing pipeline that transforms raw assignments and facts into calculated metrics:

```mermaid theme={null}
graph TB
    A[Assignment Table] --> E[Exposure Calculation]
    E --> EX[Exposure Table]
    F[Fact Table] --> J[Join Process]
    EX --> J
    J --> AG[Aggregation]
    AG --> M[Metric Results]

    style A fill:#e1f5ff
    style F fill:#e1f5ff
    style EX fill:#fff4e1
    style M fill:#e8f5e9
```

### The Join Process

Confidence processes your data through these steps:

<Steps>
  <Step title="User enters experiment">
    When a user enters an experiment, the feature flagging system assigns them to a variant. This assignment is recorded in the **Assignment Table** with a timestamp, rule, and variant identifier.

    Example:

    * User: `user7231`
    * Timestamp: `2023-05-20 05:01:21`
    * Rule: `flags/test-flag/rules/rule123`
    * Variant: `treatment`
  </Step>

  <Step title="Exposure calculation">
    Confidence queries the Assignment Table and calculates **first exposure** for each entity. This creates an **Exposure Table** that shows when each entity should start being measured.

    The exposure table filters to only include assignments for the specific experiment (using the exposure key).

    Example exposure record:

    * Entity: `user7231`
    * First Exposure: `2023-05-20 05:01:21`
    * Variant: `treatment`
  </Step>

  <Step title="User performs actions">
    The user performs actions that are tracked in the **Fact Table** (for example, makes purchases, views pages, completes tasks).

    Example facts:

    * Entity: `user7231`
    * Timestamp: `2023-05-20 12:30:00`
    * Measurement: `revenue_amount = 49.99`
  </Step>

  <Step title="Join exposure to facts">
    Confidence joins the Exposure Table to the Fact Table using the **entity identifier** (for example, `user_id`).

    The join uses the exposure timestamp to filter facts:

    * Facts must occur **after** the exposure timestamp
    * Facts must occur **within** the metric's time window (for example, 7 days after exposure), or for cumulative metrics, all facts from exposure until the calculation point

    This ensures we only measure behavior that happened after the user was exposed to the experiment.
  </Step>

  <Step title="Aggregate metrics per variant">
    After joining, Confidence aggregates the measurements:

    1. **Per-entity aggregation**: For average metrics, first aggregate facts within each entity (for example, sum all purchases per user)
    2. **Per-variant aggregation**: Then aggregate across entities within each variant (for example, average total purchases per user in treatment vs. control)

    Example:

    * Control variant: 50 users, average revenue = \$45.20
    * Treatment variant: 50 users, average revenue = \$52.30
  </Step>

  <Step title="Run statistical tests">
    Finally, Confidence runs statistical tests to determine if the difference between variants is statistically significant.

    The result shows whether the treatment variant had a meaningful impact on the metric.
  </Step>
</Steps>

### Data Flow Diagram

This diagram shows how data flows through the Confidence metrics system:

```mermaid theme={null}
graph LR
    subgraph "Data Warehouse"
        AT[Assignment Table<br/>Raw Assignments]
        FT[Fact Table<br/>Raw Events]
        EC[Exposure<br/>Calculation<br/>Runs in Warehouse]
        MC[Metric<br/>Calculation<br/>Runs in Warehouse]
        ET[Exposure Table<br/>First Exposures<br/>Cached]
        MR[Metric Results<br/>Aggregated Data<br/>Cached]
    end

    subgraph "Confidence"
        SS[Summary<br/>Statistics]
        ST[Statistical<br/>Tests]
        UI[UI<br/>Results Page]
    end

    AT --> EC
    EC --> ET
    ET --> MC
    FT --> MC
    MC --> MR
    MR --> SS
    SS --> ST
    ST --> UI

    style AT fill:#e1f5ff
    style FT fill:#e1f5ff
    style EC fill:#fff4e1
    style MC fill:#fff4e1
    style ET fill:#d4edda
    style MR fill:#d4edda
    style SS fill:#cce5ff
    style ST fill:#cce5ff
    style UI fill:#e8f5e9
```

**Key points about data flow**:

The data warehouse is either the one that [Confidence Cloud](/docs/start/confidence-cloud) runs for you, or your own with [Warehouse Native](/docs/start/warehouse-native).

1. **All raw data stays in the warehouse** - Assignment and fact tables remain in the data warehouse
2. **Calculations run in the warehouse** - Confidence executes SQL queries for exposure and metric calculations directly in the data warehouse
3. **Confidence caches intermediate results in the warehouse** - Confidence writes exposure tables and aggregated metric results to the warehouse for performance
4. **Confidence stores summary statistics** - Confidence stores aggregated summary stats and statistical test results and presents them in the UI

## Next Steps

Now that you understand how Confidence processes metrics data, you can:

* [Create an Entity](/docs/capabilities/metrics/create-entity) to define what you're measuring
* [Create a Fact Table](/docs/capabilities/metrics/create-fact-table) to connect your measurement data
* [Create an Assignment Table](/docs/capabilities/metrics/create-assignment-table) to track variant assignments
* [Create a Metric](/docs/capabilities/metrics/create-metric) to define how to aggregate your measurements

For deeper technical details, see:

* [Assignment Tables Reference](/docs/capabilities/metrics/assignment-tables)
* [Fact Tables Reference](/docs/capabilities/metrics/fact-tables)
* [Exposure Reference](/docs/capabilities/metrics/exposure)
* [Metrics Reference](/docs/capabilities/metrics/metrics)

## Related Resources

<CardGroup cols={2}>
  <Card title="Create an Entity" href="/docs/capabilities/metrics/create-entity">
    Define what you're measuring
  </Card>

  <Card title="Create a Fact Table" href="/docs/capabilities/metrics/create-fact-table">
    Connect your measurement data
  </Card>

  <Card title="Create a Metric" href="/docs/capabilities/metrics/create-metric">
    Define how to aggregate measurements
  </Card>

  <Card title="Warehouse Setup" href="/docs/start/warehouse-native/bigquery">
    Configure your data warehouse
  </Card>
</CardGroup>