Skip to main content
After creating a new metric, you should validate it to ensure it’s calculating correctly and producing expected results. The best way to validate a metric is to test it against a previous experiment where you already know the outcome.

​
Why Validate Metrics?

Validating metrics helps you:
  • Confirm the metric calculates as expected
  • Verify data joins and aggregations are correct
  • Check that the metric responds to known treatment effects
  • Inspect the underlying SQL for correctness
  • Build confidence before using the metric in production experiments

​
Before You Begin

  • A completed experiment with known results
  • A newly created metric that uses the same entity as the experiment
  • Access to create explorations

​
Validate Your Metric

1

Select a past experiment

Choose an experiment that:
  • Has already completed and produced results
  • Uses the same entity as your new metric
Navigate to the experiment’s page in Confidence.
2

Create a new exploration

  1. Go to the Explorations section in the right sidebar
  2. Click + to create a new exploration
  3. Give it a descriptive name like “Validate [Metric Name]”
3

Add your new metric

  1. Click Add metric to open the metric selection dialog
  2. Select the checkbox next to your newly created metric
  3. Click Add metrics to add it to the exploration
  4. Click Calculate to start the analysis
You can only select metrics that use the same entity configured for the experiment.
4

Add dimensions (optional)

If you want to validate how your metric behaves across different segments:
  1. Click Add dimension next to your metric
  2. Select relevant dimensions from dimension tables
  3. This helps you verify the metric works correctly across different user segments
5

Review the results

Once the calculation completes, examine the results:
  • Check the values: Do the metric values look reasonable?
  • Compare to expectations: If you know the experiment outcome, does your metric show similar patterns?
  • Review dimensions: Do dimensional breakdowns make sense?
6

Inspect the SQL

If you need to debug the calculation itself, you can inspect the SQL query that was used to calculate the metric. To do this:
  1. Find the status indicator showing the query job status
  2. Click on the status to view details
  3. Review the generated SQL query
You can copy the SQL and run it directly in your data warehouse to investigate further if needed.
7

Document your findings

Write a conclusion in the exploration describing:
  • What you were validating
  • Whether the metric behaves as expected
  • Any issues discovered and how you resolved them
  • Confirmation that the metric is ready for production use
This documentation helps others understand the validation process.

​
Common Validation Checks

When reviewing your metric results, check for:

​
Data Volume

  • Does the metric produce results?
  • Is the sample size similar to other metrics on this experiment?

​
Treatment Effect Direction

  • If the experiment had a positive effect, does your metric show that?
  • Does the magnitude seem reasonable?
  • Do confidence intervals make sense?

​
Dimensional Consistency

  • Do dimension breakdowns align with known patterns?
  • Are there any segments with suspiciously different results?

​
SQL Validation

  • Are aggregations (SUM, AVG, COUNT) applied correctly?
  • Are filters applied correctly?

Create a Metric

Learn how to create metrics in Confidence

Exploration

Understand how explorations work

Explore Results

Detailed guide on using the exploration feature

Metrics Reference

Technical reference for metrics