Skip to main content
The Confidence Agent is an AI assistant built into Confidence. Ask it in plain language to create, explain, and analyze your experiments, flags, and data. It works with the resources your account and permissions give you access to. This page lists what you can ask the agent to do, grouped by job. Each item has an example prompt and a link to the detailed page. More capabilities are coming.

​
Run Experiments and Rollouts

  • Create experiments and rollouts. The agent creates A/B tests and rollouts. It launches, ends, or archives them only after you confirm the action. Example: “Create an A/B test for the new checkout button on the web app and prepare it for launch.” See A/B tests and rollouts.
  • Change rollout reach. The agent changes the reach of a live rollout or sets an automated ramp schedule when you ask. Example: “Ramp the onboarding rollout to 10% for 3 days, then 50%.” See Understand reach.
  • Check sample size. The agent calculates the sample size an A/B test needs. Example: “How many users does the checkout test need to detect a 2% change?” See Calculate sample size.
  • Explain experiments and results. The agent summarizes an experiment’s setup and explains what its results mean. Example: “What are the results of the homepage experiment, and is it ready to ship?” See Analyze results.
  • Analyze any experiment. The agent analyzes experiments that ran in Confidence and, with Warehouse Native, experiments that ran in another tool. Example: “Analyze last month’s pricing test from the assignment table in the warehouse.” See Analyze an experiment run elsewhere.
  • Review experiment designs. Review agents check A/B tests and rollouts against your team’s instructions before launch. Example: “Request a review of the onboarding rollout.” See Use review agents.

​
Explore Your Data

  • Visualize and explore your data. The agent writes SQL against the data warehouse and turns the results into charts. Example: “Chart weekly active users by platform for the last 90 days.” See Insights.
  • Manage metric collections. The agent creates collections and adds or removes metrics, after you confirm each change. Example: “Create a collection called Checkout health with the checkout metrics.” See Manage metric collections.

​
Manage Flags

  • Manage flags. The agent finds, creates, and updates flags, variants, and targeting rules. Example: “Turn on the dark-mode flag for users in Sweden.” See Flags.
  • Test flag resolution. The agent checks which variant a flag resolves to for a given context. Example: “What does the dark-mode flag resolve to for a user in Sweden?” See Resolve tester.
  • Clean up flags. The agent archives flags you name, restores archived flags, and changes flag state. Flag changes that affect live traffic ask for your confirmation. To find flags on a client that nothing uses or that run at full rollout, use the Flags MCP tool analyzeFlagUsage. See MCP servers. Example: “Archive the flags checkout-v1 and old-banner.” See Flag cleanup.

​
Write Documents with Your Context

  • Draft product requirements in projects. Add Google Docs or Notion pages to a project, and the agent helps you write product requirements documents grounded in those documents, your data, and your experiments. Example: “Draft a product requirements document for the checkout redesign based on the attached research and last quarter’s experiments.” See Projects.

​
Agent and MCP Servers

The in-app agent and the MCP servers have different tool sets. For example, only the MCP servers create metrics. See MCP servers.

​
Where to Use the Agent