Metrics Data Processing - Confidence Documentation

Introduction

Confidence processes metrics data through four core components that work together to calculate experiment results:

Entities - The units you’re measuring (users, orders, sessions)
Fact Tables - Where your measurement data lives in the data warehouse
Assignment Tables - Where you record variant assignments
Exposure - When measurement starts for each entity

Understanding how these components interact is essential for designing effective experiments and creating accurate metrics.

The Four Components

Entities

An entity is the “thing” you’re counting or measuring in your experiments. Entities represent the units that can be uniquely identified, measured, and randomized. Key characteristic: Each entity has a unique identifier with a specific data type. Common examples:

User - Identified by user_id (string UUID or integer)
Order - Identified by order_id
Session - Identified by session_id

Entities are the foundation of your metrics system. Both your fact tables and assignment tables reference entities through their unique identifiers.

Fact Tables

Fact tables contain the events and measurements you want to analyze. These tables live in the data warehouse and describe what your entities are doing. Required structure:

SELECT
  user_id,              -- Entity identifier
  event_timestamp,      -- When the fact occurred
  revenue_amount,       -- Measurement (optional)
  platform              -- Dimension (optional)
FROM your_warehouse.events.purchases
WHERE event_timestamp BETWEEN {START_TIME} AND {END_TIME}

{START_TIME} and {END_TIME} are placeholder variables that Confidence automatically replaces with the actual time range when executing queries. These vary depending on the metric’s requirements and ensure that only relevant data within the experiment’s time window is queried.

Required columns:

entity_id - The unique identifier linking to an entity (for example, user_id, order_id)
timestamp - When the event occurred

Optional columns:

Measurements - Numeric or boolean values you want to aggregate (for example, revenue_amount, duration, is_successful)
Dimensions - Categorical values for segmentation (for example, platform, device_type)

Best practices:

Always index the timestamp column (time partitioning in BigQuery) to avoid full table scans
Index entity columns (clustering in BigQuery) to improve join performance
Remove duplicate rows to avoid double counting
Validate the distribution of nulls and zeros in your data

Assignment Tables

Assignment tables record which variant each entity was assigned to during an experiment. This data typically comes from your feature flagging system. Structure:

assignment_time	user_id	flag	rule	variant	assignment_id
2023-05-20 00:04:25	user5125	flags/test-flag	flags/test-flag/rules/rule123	flags/test-flag/variants/control	control
2023-05-20 05:01:21	user7231	flags/test-flag	flags/test-flag/rules/rule123	flags/test-flag/variants/treatment	treatment
2023-05-20 23:13:53	user3512	flags/other-test-flag	flags/other-test-flag/rules/rule123	flags/other-test-flag/variants/treatment	control

Required columns:

timestamp - When the assignment occurred (for example, assignment_time)
entity_id - The identifier of the entity that received the assignment (for example, user_id)
exposure_key - Identifies which experiment the assignment belongs to (for example, rule)
variant_key - Identifies which variant was assigned (for example, assignment_id)

Example SQL:

SELECT
  assignment_time,
  user_id,
  rule AS exposure_key,
  assignment_id AS variant_key
FROM your_warehouse.confidence.assignments
WHERE assignment_time BETWEEN {START_TIME} AND {END_TIME}

Exposure

Exposure defines when to start measuring an entity in an experiment. Confidence calculates exposure from assignment data, representing the first time an entity was assigned to a variant. Why it matters:

Ensures fair comparison between variants by measuring from a consistent starting point
Prevents bias from entities that were assigned at different times
Defines the time window for including facts in metric calculations

How Confidence calculates exposure:

Queries the assignment table for a specific experiment (filtered by exposure key)
For each entity, finds the first assignment timestamp
Groups by variant to create separate exposure cohorts
Writes the exposure table to the data warehouse for reuse

Exposure example: If user user5125 was assigned to the control variant at 2023-05-20 00:04:25, their exposure time is 2023-05-20 00:04:25. Confidence includes all facts for this user after this timestamp (within the metric’s time window) in the metric calculation.

How They Work Together

The four components form a data processing pipeline that transforms raw assignments and facts into calculated metrics:

The Join Process

Confidence processes your data through these steps:

User enters experiment

When a user enters an experiment, the feature flagging system assigns them to a variant. This assignment is recorded in the Assignment Table with a timestamp, rule, and variant identifier.Example:

User: user7231
Timestamp: 2023-05-20 05:01:21
Rule: flags/test-flag/rules/rule123
Variant: treatment

Exposure calculation

Confidence queries the Assignment Table and calculates first exposure for each entity. This creates an Exposure Table that shows when each entity should start being measured.The exposure table filters to only include assignments for the specific experiment (using the exposure key).Example exposure record:

Entity: user7231
First Exposure: 2023-05-20 05:01:21
Variant: treatment

User performs actions

The user performs actions that are tracked in the Fact Table (for example, makes purchases, views pages, completes tasks).Example facts:

Entity: user7231
Timestamp: 2023-05-20 12:30:00
Measurement: revenue_amount = 49.99

Join exposure to facts

Confidence joins the Exposure Table to the Fact Table using the entity identifier (for example, user_id).The join uses the exposure timestamp to filter facts:

Facts must occur after the exposure timestamp
Facts must occur within the metric’s time window (for example, 7 days after exposure), or for cumulative metrics, all facts from exposure until the calculation point

This ensures we only measure behavior that happened after the user was exposed to the experiment.

Aggregate metrics per variant

After joining, Confidence aggregates the measurements:

Per-entity aggregation: For average metrics, first aggregate facts within each entity (for example, sum all purchases per user)
Per-variant aggregation: Then aggregate across entities within each variant (for example, average total purchases per user in treatment vs. control)

Example:

Control variant: 50 users, average revenue = $45.20
Treatment variant: 50 users, average revenue = $52.30

Run statistical tests

Finally, Confidence runs statistical tests to determine if the difference between variants is statistically significant.The result shows whether the treatment variant had a meaningful impact on the metric.

Data Flow Diagram

This diagram shows how data flows through the Confidence metrics system: Key points about data flow: The data warehouse is either the one that Confidence Cloud runs for you, or your own with Warehouse Native.

All raw data stays in the warehouse - Assignment and fact tables remain in the data warehouse
Calculations run in the warehouse - Confidence executes SQL queries for exposure and metric calculations directly in the data warehouse
Confidence caches intermediate results in the warehouse - Confidence writes exposure tables and aggregated metric results to the warehouse for performance
Confidence stores summary statistics - Confidence stores aggregated summary stats and statistical test results and presents them in the UI

Next Steps

Now that you understand how Confidence processes metrics data, you can:

Create an Entity to define what you’re measuring
Create a Fact Table to connect your measurement data
Create an Assignment Table to track variant assignments
Create a Metric to define how to aggregate your measurements

For deeper technical details, see:

Create an Entity

Define what you’re measuring

Create a Fact Table

Connect your measurement data

Create a Metric

Define how to aggregate measurements

Warehouse Setup

Configure your data warehouse

​Introduction

​The Four Components

​Entities

​Fact Tables

​Assignment Tables

​Exposure

​How They Work Together

​The Join Process

​Data Flow Diagram

​Next Steps

​Related Resources

Create an Entity

Create a Fact Table

Create a Metric

Warehouse Setup

Introduction

The Four Components

Entities

Fact Tables

Assignment Tables

Exposure

How They Work Together

The Join Process

Data Flow Diagram

Next Steps

Related Resources