Overview
The review process for A/B tests and rollouts can happen in several ways in Confidence:- You can request a review from a team mate by clicking the plus icon in the Reviews section on the sidebar.
- Anyone that has read rights to your experiment can decide to leave a review by clicking the thumbs-up-and-down icon in the Reviews section on the sidebar.
- A surface that you run your experiment on can suggest or require reviewers on your experiment. If your experiment runs on surfaces that require reviewers, you can’t launch the experiment unless at least one reviewer from each surface has approved the experiment.
Get pinged on Slack when someone mentions you or replies to your comment
by connecting your personal Slack account with
Confidence.
Required and Optional Reviewers
All reviewers that you add on the experiment design page are optional in the sense that you can launch the experiment without approval. Required reviewers only come from surfaces.Experiment Changes After Approval
Changes to approved experiments reset the approvals. This means that required reviewers must approve the experiment again before you can launch it. Changes that reset the approvals are:- Changing the flag, flag variants, or adding more variants.
- Changing any settings in the audience section, including the allocation percentage.
Add Reviewers for a Surface
The surface owner can add suggested or required reviewers to the surface. Configure reviewers on the surface settings page.Request a Review
Request a review from a team mate by clicking the plus sign icon in the Reviews section on the right sidebar of the experiment design page. Reviewers can approve or reject the experiment design and leave comments. Request reviews from surface reviewers by clicking Request review next to their name. To request a review from all reviewers on a surface, click Request review next to the surface name. Reviews from optional reviewers are only informational and do not block launching experiments. You should discuss in your team on how to work with reviews.If a reviewer requests your review for an experiment, there is a
banner on top of the experiment page when you visit the experiment to remind
you to leave your review. This banner disappears for already approved
experiments.
Review an Experiment
To review an experiment, click the comment icon at the top right corner on the experiment page. After you have added comments on sections you want to discuss or request changes to, submit the review by making an overall judgment on the experiment design and approve or reject it. To review an experiment:- Go to an A/B test or a rollout.
- Click the thumbs-up-and-down icon at the top of the Reviews section on the right sidebar.
- Select the outcome of the review in the review dialog. Add a comment if needed.
- Click Save.