Welcome to Intro to metrics
Intro to metrics is an asynchronous, self-paced course that teaches you the fundamental concepts of metric design and selection. This course focuses on building intuition for what makes a good metric and how to design metrics that drive better product decisions.
In this course, you learn how to define metrics that capture the right user behavior, choose appropriate measurement approaches, and understand the different types of metrics used in experimentation and product development. Whether you're evaluating experiments, tracking product performance, or setting team goals, this course provides the foundation for working effectively with metrics.
There are quiz questions throughout the course to help you check your understanding of the material. Complete each lesson's questions to track your progress.
Before you begin
Before you start this course, you should go through the A/B test quickstart to familiarize yourself with the basics of running experiments in Confidence. While this course is platform-agnostic, understanding the experiment workflow helps you see how metrics fit into the broader context of product development and experimentation.
Lessons
This course consists of the following lessons:
Lesson 1: What is a metric?
Define metrics and understand their role in product decisions and experiments.
Lesson 2: Metric roles
Understand success, guardrail, exploratory, and diagnostic metric roles.
Lesson 3: Time considerations
Choose appropriate time windows for short-term and long-term impacts.
Lesson 4: Capturing behavior
Design metrics that capture behavior without gaming or unintended effects.
Lesson 5: Strategic metrics
Understand KPIs, proxy metrics, and the strategic metric hierarchy.
Lesson 6: Interpretability
Create understandable metrics with clear naming and documentation.
Lesson 7: Feasibility and sensitivity
Evaluate feasibility, variance, and influenceability for experiments.
Lesson 8: Variance reduction
Understand regression adjustment, how much variance reduction to expect, and when to cap.
Lesson 9: Select metrics
Apply your learnings from previous lessons and practice selecting a complete metric suite.
Lesson 10: Segment-level analysis
Break down experiment results by user segments to find meaningful patterns across groups.