Skip to content
Confidence
  • Documentation
  • Blog
  • Bootcamp
  • Status
  • Confidence Bootcamp
    • My learning
    • Intro to experimentation
      • Introduction
      • Lesson 1: Why you should experiment
      • Lesson 2: Experiment hypothesis
      • Lesson 3: Success and guardrail metrics
      • Lesson 4: Success metrics
      • Lesson 5: Set up your experiment
      • Lesson 6: Calculation frequency
      • Lesson 7: Target audience
      • Lesson 8: Sample size
      • Lesson 9: Quality assurance
      • Lesson 10: Run your experiment
      • Lesson 11: Evaluate your experiment and make a decision
      • Lesson 12: A/B tests and rollouts
      • Course wrap up
    • Intro to metrics
      • Introduction
      • Lesson 1: What is a metric?
      • Lesson 2: Metric roles
      • Lesson 3: Time considerations
      • Lesson 4: Capturing behavior
      • Lesson 5: Strategic metrics
      • Lesson 6: Interpretability
      • Lesson 7: Feasibility and sensitivity
      • Lesson 8: Variance reduction and metric selection
      • Lesson 9: Select metrics
      • Lesson 10: Segment-level analysis
      • Course wrap up
    • Scientific product development
      • Introduction
      • Lesson 1: Why you should experiment
      • Lesson 2: The scientific method
      • Lesson 3: Randomized controlled trials
      • Lesson 4: Experiment hypothesis
      • Lesson 5: Case study
        • Case study
        • Answers to case study
      • Lesson 6: Why do we need statistics?
      • Lesson 7: Success metrics
      • Lesson 8: Detectable effects and sample size
      • Lesson 9: Make a decision
      • Course wrap up
    • A primer on hypothesis testing
      • Introduction
      • Lesson 1: Introduction to hypothesis testing
      • Lesson 2: True vs estimated effects
      • Lesson 3: Sampling distribution of the difference-in-means estimator
      • Lesson 4: Z-tests and how to reject the null hypothesis
      • Lesson 5: False postive rate and alpha
      • Lesson 6: True positive rate, MDE, and power
      • Course wrap up
    • Intro to Feature Flags
      • Introduction
      • Lesson 1: What is a feature flag?
      • Lesson 2: Lifecycle of a feature flag
      • Lesson 3: Clients
      • Lesson 4: Evaluation context and targeting
    • Sample size calculation - I
      • Introduction
      • Lesson 1: What is the required sample size?
      • Lesson 2: Alpha and power
      • Lesson 3: Baseline mean and variance
      • Lesson 4: Sample size playground - I
    • Sample size calculation - II
      • Introduction
      • Lesson 1: Multi-metric decision making
      • Lesson 2: Number of success metrics
      • Lesson 3: Number of guardrail metrics
      • Lesson 4: Number of comparisons
      • Lesson 5: Sample size playground - II
    • Sample size calculation - III
      • Introduction
      • Lesson 1: Binary metrics
      • Lesson 2: Treatment group proportions
      • Lesson 3: Variance reduction
      • Lesson 4: Sequential testing and sample size
      • Lesson 5: Sample size playground - III
    • Advance your experimentation
      • Introduction
      • Lesson 1: Guardrail metrics with non-inferiority margins
      • Lesson 2: Choose evaluation frequency
      • Lesson 3: Metrics' roles in experiments
      • Lesson 4: Cumulative holdback evaluations
    • Experimentation culture
      • Introduction
      • Lesson 1: Onboarding into experimentation
      • Lesson 2: Empowering experimentation champions
      • Lesson 3: Sustaining the experimentation culture
    • Videos

Welcome to Intro to metrics

Summary

A self-paced course on metric design: metric roles, time windows, capturing behavior, strategic metrics, sensitivity, and selecting a metric suite.

Intro to metrics is an asynchronous, self-paced course that teaches you the fundamental concepts of metric design and selection. This course focuses on building intuition for what makes a good metric and how to design metrics that drive better product decisions.

In this course, you learn how to define metrics that capture the right user behavior, choose appropriate measurement approaches, and understand the different types of metrics used in experimentation and product development. Whether you're evaluating experiments, tracking product performance, or setting team goals, this course provides the foundation for working effectively with metrics.

Note

There are quiz questions throughout the course to help you check your understanding of the material. Complete each lesson's questions to track your progress.

Before you begin

Before you start this course, you should go through the A/B test quickstart to familiarize yourself with the basics of running experiments in Confidence. While this course is platform-agnostic, understanding the experiment workflow helps you see how metrics fit into the broader context of product development and experimentation.

Lessons

This course consists of the following lessons:

Lesson 1: What is a metric?

Define metrics and understand their role in product decisions and experiments.

Not completed

Lesson 2: Metric roles

Understand success, guardrail, exploratory, and diagnostic metric roles.

Not completed

Lesson 3: Time considerations

Choose appropriate time windows for short-term and long-term impacts.

Not completed

Lesson 4: Capturing behavior

Design metrics that capture behavior without gaming or unintended effects.

Not completed

Lesson 5: Strategic metrics

Understand KPIs, proxy metrics, and the strategic metric hierarchy.

Not completed

Lesson 6: Interpretability

Create understandable metrics with clear naming and documentation.

Not completed

Lesson 7: Feasibility and sensitivity

Evaluate feasibility, variance, and influenceability for experiments.

Not completed

Lesson 8: Variance reduction and metric selection

Understand regression adjustment, how much variance reduction to expect, and when to cap.

Not completed

Lesson 9: Select metrics

Apply your learnings from previous lessons and practice selecting a complete metric suite.

Not completed

Lesson 10: Segment-level analysis

Break down experiment results by user segments to find meaningful patterns across groups.

Not completed

Was this page helpful?

PreviousCourse wrap up
NextLesson 1: What is a metric?

© Copyright 2026. All rights reserved.

Follow us on TwitterFollow us on GitHub

On this page

  1. Before you begin

  2. Lessons