Skip to main content

​
Symptoms

You’re experiencing this if:
  • Exposure calculated correctly (non-zero exposure available for the experiment)
  • Metric calculation job completed successfully
  • Metric results delivered check shows an error

​
Common Causes and Diagnoses

​
Cause 1: Fact Table Returns No Data

​
Explanation

If your fact table query returns 0 rows, the problem is in your fact table query.

​
Diagnosis

1

Find your fact table query

  1. Navigate to the Design tab of your experiment
  2. Go to the metric you want to troubleshoot
  3. Select the Fact table
  4. Copy the fact table query
2

Run your fact table query

  1. Paste the fact table query into your data warehouse query editor (Warehouse Native only)
  2. Run the query
  3. If the query returns 0 rows, the problem is in your fact table query
If it returns data, add a filter to your query to only include facts for the period your metric is calculating for to verify that there is data also for the relevant period.

​
Cause 2: Mismatch Entity IDs Between Fact Table and Assignment Table

​
Explanation

If your fact table query returns data but the entity IDs in the fact table and the assignment table don’t match, Confidence cannot find the exposed users in the fact table.

​
Diagnosis

If your fact table query returns data but the metric shows No data, a possible reason is that the entity IDs in the fact table and the assignment table don’t match. For example, if the fact table and assignment table use the same user IDs but the user IDs are in different formats, or truncated differently. Investigate by comparing the entity IDs in the fact table with the entity IDs in the assignment table.

​
Cause 3: Empty Partitions Committed

​
Explanation

Confidence waits for data to arrive in fact table partitions. If the data delivery cadence is set to Data is partitioned hourly or Data is partitioned daily, partitions may be committed empty depending on your settings.

​
Diagnosis

1

Check data delivery cadence

  1. Navigate to the Design tab of your experiment
  2. Select the fact table in the Metric config section
  3. Check Data delivery cadence setting to see if it is Daily or Hourly
2

Check max zero rows duration

  1. On the fact table configuration page, select the edit icon in the Configuration section on the right sidebar
  2. If the Disable empty partitions option is enabled, Confidence never commits empty partitions.
  3. If the Disable empty partitions option is disabled, Confidence commits empty partitions if data doesn’t arrive within the configured window defined by the Max zero rows duration setting.

​
Example

Example scenario for a daily partitioned fact table:

​
Cause 4: Metric Has No Variance

​
Explanation

If metrics have data, but all the data is the same value, the metric has no variance. This leads to the Metric data delivered check showing an error, because Confidence is unable to use the metric for statistical analysis.

​
Diagnosis

1

Check metric results delivered check

  1. Navigate to the Results tab of your experiment
  2. Check the Metric results delivered check to see if it shows an error and if the error is that the metric has no variance.
  3. Click on the metric’s calculation status in the Metric calculations section on the right sidebar to see the details of the calculation.
  4. In the logs section, expand a calculation entry to view the summary statistics for the metric. Here you can see the variance of the metric as well as the mean and the count.
A metric can have no variance because while there is some data for the metric, all the data is the same value. For example, every user had purchase_amount equal to 0 for the calculation period. Equal values are more likely when there are only a few users in the calculation. The issue can also occur if the metric is misconfigured. For example, if you configured the metric to use thresholds and no users met the threshold, or if the metric uses filters that are not set up correctly.

Metrics Introduction

Overview of metrics in Confidence

Fact Tables

Configure fact tables

Exposure Calculation Stuck

Troubleshoot exposure issues