Skip to main content
Many websites serve both unauthenticated and authenticated users. A common situation in experimentation is that you want to randomize unauthenticated users, but follow the same users and measure their behavior after they authenticate. This type of A/B testing is popular when experimenting on sign-up flows. For example, you make changes to the sign-up flow and measure the impact on engagement when authenticated. To run an experiment like this, you need to be able to:
  1. randomize users that haven’t been assigned an identifier yet
  2. describe the relation between unauthenticated and authenticated users
  3. ensure that an unauthenticated user gets the same experience after authenticating.

​
Randomize on Unauthenticated Users

You can randomize on unauthenticated users by having a required cookie, which is a website cookie that’s essential for the site to work. The first time a user visits the site you can generate a UUID as an identifier for the user, and store it in a required cookie. You then pass this identifier to Confidence and use it as the randomization unit in your experiments. The user may still delete the cookie, get a new identifier, and potentially be present in multiple treatments resulting in biased estimates.

​
Measure Unauthenticated and Authenticated Users in the Same Experiment

You can measure both unauthenticated users and authenticated users by adding an entity relationship. The relationship maps the IDs of unauthenticated users to the IDs of authenticated users. After you have set up an entity relationship you can add metrics for both authenticated users and unauthenticated users. For example, the entity relationship allows you to measure user behavior both before and after sign-up. Read more about entity relationships.
Exposure is the timestamp of the first assignment of the unauthenticated identifier. Metrics for authenticated users measure behavior relative to the exposure timestamp of the unauthenticated identifier the entity-relation table maps the user to.

​
Consistent Experience After Sign-up

You can ensure that a user gets a consistent experience after authenticating by continuing to pass the cookie even after the user has authenticated. This way all rules that target the cookie continue to match even after authentication, and any rule that targets the authenticated ID can now randomize on that instead.

Entity Relation Tables

Connect unauthenticated to authenticated users

Create Entity Relations

Step-by-step entity relation guide

Entities Reference

Configure entity types

Audience Reference

Configure randomization settings