> ## Documentation Index
> Fetch the complete documentation index at: https://confidence.spotify.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Metrics
> Metrics define what to evaluate an experiment on.
The success of an experiment depends on how the metrics you select move.
Metrics describe the behavior of the entity you're experimenting on.
By first calculating the metrics for the control and treatment groups, you can later statistically
compare the two groups.
Metrics in Confidence belong to one of two classes of metrics: success or guardrail metrics.
You first need to create a metric before you can use it in an experiment.
You can read more about how to create metrics on the [metrics](./metrics) page.
## Assignments
For Confidence to be able to evaluate the experiment, it needs to know what you are experimenting on.
Specify an entity, and where the assignments for these entities exist.
1. Click the **Edit** icon to the right on the **Metrics** section of the experiment edit page to bring up
the metric configuration dialog.
2. Select the entity that you want to analyze.
3. Select the assignment table that has assignment logs for the experiment.
4. Optional. Select an [exposure filter](#exposure-filtering).
5. Select how often you want to compute metrics by specifying the metric interval.
6. Click **Save** to save the metric configuration.
### Exposure Filtering
Exposure filters are methods for narrowing down more closely which users
to include in the exposure definition and the analysis of your experiment.
When you add an exposure filter for your experiment, the analysis only
includes the exposed users that also match the exposure filter. The time of
exposure is the first unit of time after default exposure where the user
matches the exposure filter.
You may want to use exposure filters if the default definition of exposure is
too broad for what you want to measure in your experiment.
[Read more about Exposure Filtering](/docs/metrics/exposure)
## Success Metrics
Success metrics aim to prove the hypothesis of the A/B test. For example,
if your hypothesis is that "more users stream podcasts if you rank
podcasts higher," then an appropriate success metric is a metric that measures podcast
consumption.
In this example, the hypothesis expects the metric to increase.
A significant result means that there is evidence of an effect in the desired direction.
Test for the following change in a success metric.
Test for a significant increase of the metric. For example, an
increase in hours spent listening to Spotify.
Test for a significant decrease of the metric. For example, a
decrease in the number of skipped songs in personalized playlists.
You can optionally set a [minimum detectable effect
(MDE)](./design/effect-sizes#minimum-detectable-effects) for success metrics. The MDE is the effect
size that you want to be able to detect with high certainty. You must set an MDE to be able to run a
[power analysis](/docs/experiments/statistical-settings) for the metric and
learn how much traffic the metric needs.
## Guardrail Metrics
A guardrail metric is a metric that ensures that the experiment doesn't have any
unexpected side effects.
You can use any metric as a guardrail metric.
The meaning of a significant result depends on the type of guardrail metric.
Use guardrail metrics in the following situations:
* When you want to ensure that your A/B test doesn't introduce regressions in
performance or product quality.
* When you want to ensure that your A/B test doesn't have a negative impact on
a metric that some other part of the organization cares about.
Guardrail metrics can use non-inferiority margins that let you seek evidence that the change doesn't
have a negative impact that exceeds the margin you specify. The margin is optional. The interpretation
of the results depends on whether you use non-inferiority margins. All guardrail metrics in the same
experiment must either all use or all not use non-inferiority margins.
### With Non-Inferiority Margin
Use guardrail metrics with non-inferiority margins to look for evidence that the change doesn't
negatively impact the metric more than your specified non-inferiority margin.
A significant result means that there is evidence that the guardrail is within acceptable margins.
Guardrail metrics with non-inferiority margins test for the following.
Test if there is evidence that the metric hasn't increased
by more than the NIM. For example, the number of skipped songs in personalized playlists shouldn't
increase by more than 1%.
Test if there is evidence that the metric hasn't decreased
by more than the NIM. For example, the number of hours spent listening to Spotify shouldn't
decrease by more than 1%.
### Without Non-Inferiority Margin
Use guardrail metrics without non-inferiority margins to look for evidence that the change has a
negative impact on the metric. A significant result means that there is evidence that the guardrail
deteriorates because of the change.
Guardrail metrics with non-inferiority margins test for the following.
Test if there is evidence that the metric has increased. For example, test if the number of skipped
songs in personalized playlists has increased.
Test if there is evidence that the metric has decreased. For example, test if the number of hours
spent listening to Spotify has decreased.
Set non-inferiority margins if you can. With non-inferiority margins, you seek evidence that the change doesn't
lead to a regression by more than a tolerance level you specify. Without a non-inferiority margin, a
lack of evidence of a deterioration doesn't imply a neutral result. Read more about how the two
approaches compare in [the guardrail metric lesson](/docs/experiments/metrics).
## Required Metrics
For experiments that use a surface with required metrics,
Confidence adds these metrics to the bottom of the design page. Read more about
required metrics in the [surface documentation](/docs/surfaces/introduction#required-metrics).
## Related Resources
Step-by-step metric setup
Deep dive into metric configuration
Configure MDE and NIM settings