> ## Documentation Index
> Fetch the complete documentation index at: https://confidence.spotify.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Use Review Agents

> Learn how to create and use AI review agents that provide automated feedback on experiment designs.

export const HowToSchema = ({name, description, steps = []}) => {
  const schema = {
    "@context": "https://schema.org",
    "@type": "HowTo",
    name,
    description
  };
  if (steps.length > 0) {
    schema.step = steps.map((s, i) => ({
      "@type": "HowToStep",
      position: i + 1,
      name: typeof s === "string" ? s : s.name,
      text: typeof s === "string" ? s : s.text || s.name
    }));
  }
  return <script type="application/ld+json" dangerouslySetInnerHTML={{
    __html: JSON.stringify(schema)
  }} />;
};

<HowToSchema
  name="Use Review Agents"
  description="Learn how to create and use AI review agents that provide automated feedback on experiment designs."
  steps={[
{
  name: "Go to Agents",
  text: "Go to Confidence and select Admin on the left sidebar, then select Agents.",
},
{
  name: "Create agent",
  text: "Click Create and enter a name and description for your agent.",
},
{
  name: "Enable Review skill",
  text: "Toggle on the Review skill to allow the agent to review experiments.",
},
{
  name: "Add instructions",
  text: "Enter instructions that define how the agent should review experiments. See Write effective instructions for guidance.",
},
{ name: "Save", text: "Click Save to create your agent." },
{ name: "Open agent", text: "Go to Admin > Agents and select your agent." },
{ name: "Start preview", text: "Click Preview." },
{
  name: "Select experiment",
  text: "Select a draft A/B test or rollout from the dropdown list.",
},
{
  name: "Run review",
  text: "Click Review to see how the agent would review the experiment.",
},
{ name: "Open experiment", text: "Go to an A/B test or rollout." },
{
  name: "Add reviewer",
  text: "In the Reviews section on the right sidebar, click the plus icon.",
},
{ name: "Select agent", text: "Search for and select the review agent." },
{ name: "Request review", text: "Click Request next to the agent name." },
]}
/>

<Badge color="orange">Beta</Badge>

<Tip>
  AI review agents are currently in beta. If you don't see the **Agents** page in
  your organization, contact
  [experimentation-cs@spotify.com](mailto:experimentation-cs@spotify.com) to
  request access.
</Tip>

Create an AI review agent to provide automated feedback on A/B test and rollout
designs. Agents analyze experiment configurations and post comments on
individual sections based on your instructions.

## Create the Agent

<Steps>
  <Step title="Go to Agents">
    Go to Confidence and select **Admin** on the left sidebar, then select
    **Agents**.
  </Step>

  <Step title="Create agent">
    Click **Create** and enter a name and description for your agent.
  </Step>

  <Step title="Enable Review skill">
    Toggle on the **Review** skill to allow the agent to review experiments.
  </Step>

  <Step title="Add instructions">
    Enter instructions that define how the agent should review experiments.
    See [Write effective instructions](#write-effective-instructions) for
    guidance.
  </Step>

  <Step title="Save">
    Click **Save** to create your agent.
  </Step>
</Steps>

## Write Effective Instructions

Structure your instructions by section to help the agent provide targeted
feedback. The agent reviews different sections depending on the experiment type.
Map your instructions to the sections on the experiment Design page.

### A/B Test Sections

| Section          | What to define                                                                                |
| ---------------- | --------------------------------------------------------------------------------------------- |
| **Display name** | Naming conventions or required prefixes                                                       |
| **Hypothesis**   | What makes a good hypothesis statement                                                        |
| **Flag**         | Flag selection and configuration expectations                                                 |
| **Treatments**   | Requirements for treatment descriptions, images, and allocation                               |
| **Audience**     | Targeting criteria and allocation expectations                                                |
| **Surfaces**     | Surface selection requirements                                                                |
| **Metrics**      | Metric selection guidelines, role assignments, MDE and NIM requirements, preferred directions |
| **Stats**        | Alpha, power, test horizon strategy, and exposure filter expectations                         |
| **Sample size**  | Sample size calculation requirements                                                          |
| **Planning**     | Experiment duration and scheduling expectations                                               |
| **Links**        | Required documentation or resources                                                           |

### Rollout Sections

| Section               | What to define                                        |
| --------------------- | ----------------------------------------------------- |
| **Display name**      | Naming conventions or required prefixes               |
| **Description**       | What the rollout description should include           |
| **Feature**           | Variant selection and initial reach expectations      |
| **Audience**          | Targeting criteria and allocation expectations        |
| **Surfaces**          | Surface selection requirements                        |
| **Metrics**           | Monitoring metric selection guidelines                |
| **Stats**             | Alpha and power expectations                          |
| **Sample size**       | Sample size calculation requirements                  |
| **Planning**          | Rollout duration and scheduling expectations          |
| **Automatic ramp-up** | Ramp-up schedule, step count, and timing expectations |
| **Links**             | Required documentation or resources                   |

<Tip>
  Be specific about what the agent should check. For example, instead of "check
  the hypothesis," write "verify the hypothesis states a clear expected outcome
  with a measurable effect on the primary metric."
</Tip>

## Preview Agent Reviews

Test how your agent reviews experiments before using it on real experiments.

<Steps>
  <Step title="Open agent">
    Go to **Admin** > **Agents** and select your agent.
  </Step>

  <Step title="Start preview">
    Click **Preview**.
  </Step>

  <Step title="Select experiment">
    Select a draft A/B test or rollout from the dropdown list.
  </Step>

  <Step title="Run review">
    Click **Review** to see how the agent would review the experiment.
  </Step>
</Steps>

The preview shows the agent's overall judgment (approved or rejected) along
with feedback for each section. Use this to refine your instructions.

## Request an Agent Review

Request a review from an agent on the experiment Design page.

<Steps>
  <Step title="Open experiment">
    Go to an A/B test or rollout.
  </Step>

  <Step title="Add reviewer">
    In the **Reviews** section on the right sidebar, click the plus icon.
  </Step>

  <Step title="Select agent">
    Search for and select the review agent.
  </Step>

  <Step title="Request review">
    Click **Request** next to the agent name.
  </Step>
</Steps>

The agent analyzes the experiment and posts comments on individual sections
where it has feedback.

<Note>
  AI agent reviews are informational and don't count toward required reviews.
  An AI agent approval does not meet the required reviewer requirement for
  launching—at least one human reviewer must approve the experiment.
</Note>

## Related Resources

<CardGroup cols={2}>
  <Card title="Reviews" href="/docs/experiments/reviews">
    Learn about the review process
  </Card>

  <Card title="Configure Surface Reviews" href="/docs/how-to-guides/configure-surface-reviews">
    Set up required reviewers for surfaces
  </Card>
</CardGroup>