Skip to main content
To measure the impact of a change, you need to know when a user was first exposed to the change. This time of exposure is the timestamp of the first assignment for a user.

​
How Exposure Calculations Work

Exposure calculations use raw assignment data stored in an Assignment Table. The exposure calculations use four inputs:
  1. An assignment table. Specifies what assignment table in the data warehouse to query.
  2. A start and stop time. The times define what interval to query the assignment table for.
  3. Exposure Key. The exposure calculation filters rows in the assignment table where the exposure key is equal to the supplied value.
  4. Variant Key. The calculation filters rows in the assignment table where the variant key is equal to the given value. If you don’t specify a variant key, the calculation doesn’t filter on the variant key.

​
Storage

Exposure calculations write to your data warehouse. You configure the destination schema and dataset when you set up the data warehouse connection. The exposure calculations create a new table in the data warehouse for all new A/B tests and rollout.

​
Exposure Time

Time of exposure is the timestamp of the first assignment for a user. Later assignments don’t affect the exposure calculation.

​
Exposure Filtering

Exposure filters are also referred to as ‘trigger analysis’ in the literature and blogs.
Add an exposure filter on the experiment design page by clicking Results on the left sidebar, then Exposure filters. You can add several exposure filters per experiment. To keep interpretation of the results clear, use one or two exposure filters. Confidence produces metric results for each exposure filter (including no filter) separately. You can see the results for any exposure filter (including no filter) on the results page by selecting the corresponding filter in the top of the results page.
You can see the results for all metrics with and without all exposure filters simultaneously by clicking Detailed results in the top right corner of the metrics result section in the results page. Confidence shows the exposure filter column by default if the experiment has any exposure filters.
Exposure filters are methods for narrowing down more closely which users to include in the exposure definition and the analysis of your experiment. When you add an exposure filter for your experiment, the analysis only includes the exposed users that also match the exposure filter. The time of exposure is the first unit of time after default exposure where the user matches the exposure filter. You may want to use exposure filters if the default definition of exposure is too broad for what you want to measure in your experiment.
Exposure filters don’t affect who actually gets the experiment experience. The filter only narrows down who counts as exposed in your A/B test. Since the exposure filter event might happen some time after the base exposure event, an exposure filter also affects when in time exposure happens, which in turn determines who to include in the experiment results at different points in time. For example, the exposure count without an exposure filter might be 1000 at a given time point, but the exposure count with an exposure filter might be 500 for the same time point. If you would like to limit who actually sees the A/B test, you should instead use inclusion criteria.
You run an A/B test where you add credits and links at the bottom of Spotify’s Rap Caviar playlist. Possible exposure definitions that range from less to more restrictive include:
  1. Launching the app
  2. Opening a playlist
  3. Opening Rap Caviar
  4. Playing from the Rap Caviar playlist
  5. Scrolling to the bottom of the Rap Caviar playlist
Which exposure definition is appropriate depends on the purpose of the A/B test.
To set up an exposure filter, you first need to create a fact table that has information about the exposure criteria. For example, all users who visited a certain page. You can then use this fact table to filter the exposure of the experiment. When you add an exposure filter, experiment results include only users that are both in the default exposure table and in the exposure filter table. This filtering means that the analysis only includes users that fulfill the exposure criteria, defined as being present in the fact table.

​
Filter the Exposure Filter Fact Table

Select rows from the fact table you use as an exposure filter by filtering the rows on values from columns in that fact table. If you for example have several types of page views events in the same fact table, you can filter out only the page views from a certain page to use as the exposure filter.
For exposure filtering, you can only select fact tables that have a column of the type Entity that matches the entity for the experiment.

​
Health Checks with Exposure filters

Confidence runs all health checks with and without all exposure filters and sends alert to the experiment owner if a health check fails.

​
Schedules

When you run an experiment you typically don’t want to calculate exposure just once but multiple times throughout the runtime of the experiment. This recurring calculation is typically referred to as an Exposure Schedule in Confidence. Although Confidence supports any variable intervals the built-in workflows come in two modes: hourly and daily. For both of these modes the actual intervals are variable and slowly ramp up to the interval you’ve selected. This ensures that you get quick feedback as soon as you’ve launched your experiment. Here’s an example of how an hourly schedule might look if launched at 12:46:45.
  1. 3 minutes 15 seconds (12:46:45 to 12:50). This ensures that the schedule aligns with even minutes.
  2. 5 minutes (12:50 to 12:55).
  3. 10 minutes (12:55 to 13:05).
  4. 15 minutes (13:05 to 13:20).
  5. 40 minutes (13:20 to 14:00).
  6. 60 minutes (14:00 to 15:00). The schedule remains at 60 minutes after reaching the last interval.
The built-in workflows manage your schedules for you.

Use Exposure Filters

Step-by-step exposure filter guide

Assignment Tables

Configure assignment data sources

Create Fact Tables

Set up exposure filter data sources

Interpreting Results

Understand experiment results