Skip to main content
This tutorial helps you configure Confidence to:
  1. Run queries in Redshift to compute exposure and metrics.
  2. Store assignment data as Parquet files in S3, and then load them into Redshift.
Step two (2) is optional if you already have assignment data in Redshift. For example, if you are using a feature flagging solution other than Confidence Flags. This document targets the following audiences:
  • Administrators who want to set up Confidence for their organization

​
Before You Begin

  • You need to have a Confidence account.
  • You need to have a AWS account.
  • You need to have permissions to create S3 buckets, IAM users and roles, and manage Redshift instances.

​
Terraform Scripts

Terraform scripts are available to create the S3 bucket, IAM user and role described in the steps below. If you are using Terraform, then after you have applied the Terraform Configuration, you can skip directly to step Step 3.

​
Step 1: Create a S3 Bucket

To load assignment data, Confidence first copies Parquet files to a S3 bucket, and then triggers load jobs to copy these into Redshift.
  • Go to the S3 console, click Create bucket.
  • Give it a name, and put it in the same AWS region as you have your Redshift instance in.

​
Step 2: Create the Confidence IAM role

Now you need to create an IAM role that Confidence can assume with the correct permissions. Two options for authentication are available. Either Confidence can use a regular AWS access key and secret to authenticate as an IAM User and then assume the role, or it can use AssumeRoleWithWebIdentity to authenticate without having to store any credentials, by using a Google service account as the trusted entity. AssumeRoleWithWebIdentity is usually preferable, but sometimes it might interfere with other settings such as custom identity providers. In those cases, you may need the credentials-based approach. Do either step 2a or 2b depending on what approach you choose.

​
Step 2a: Setup the Trust policy for AssumeRoleWithWebIdentity

  • Go to the IAM console, click Roles and Create role
  • Select “Custom trust policy” as the trusted entity type.
  • In the text field, paste the following JSON snippet, replacing <service_account_id> with the unique service account ID you are using to authenticate from the Confidence side. You can find the ID in the Your Service Account ID box that is part of the configure data warehouse form for Redshift.
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "redshift.amazonaws.com",
          "redshift-serverless.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    },
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "accounts.google.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "accounts.google.com:sub": "<service_account_id>"
        }
      }
    }
  ]
}
  • Click next, and don’t select any of the predefined permissions. Confidence adds its own inline policy that is more restrictive than the built-in policies.
  • Input a name for the role, for example, confidence-role, and then click Create role.

​
Step 2b: Setup the Trust Policy with an IAM User

  • Go to the IAM console, click Users and Create user
  • Give the user a name and create it.
  • Go to the user details and generate an access key and secret for the user. Keep the access key and secret for later when you configure the warehouse in Confidence.
  • Go to the IAM console, click Roles and Create role
  • Select “Custom trust policy” as the “Trusted entity” type.
  • In the text field, paste the following JSON snippet, replacing <user_arn> with the ARN of the user you created in step 2 above (there is a button to copy the ARN on the user page).
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "redshift-serverless.amazonaws.com",
          "redshift.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    },
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "<user_arn>"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

​
Step 2c: Setup the IAM Role Policy

  • Find the role you created earlier and select it, then click the Add permission dropdown list and then Create inline policy
  • Switch the policy editor to JSON, and then paste the following snippet, replacing the place holders with the name of the S3 bucket you created, your AWS region, AWS account ID and the Redshift cluster name.
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "s3:ListBucket",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::<s3_bucket_name>"
    },
    {
      "Action": [
        "s3:PutObjectAcl",
        "s3:PutObject",
        "s3:GetObjectAcl",
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::<s3_bucket_name>/*"
    },
    {
      "Action": [
        "redshift-data:ListStatements",
        "redshift-data:GetStatementResult",
        "redshift-data:DescribeStatement",
        "redshift-data:CancelStatement"
      ],
      "Effect": "Allow",
      "Resource": "*"
    },
    {
      "Action": [
        "redshift:GetClusterCredentialsWithIAM",
        "redshift:GetClusterCredentials",
        "redshift-data:ListTables",
        "redshift-data:ListSchemas",
        "redshift-data:ListDatabases",
        "redshift-data:ExecuteStatement",
        "redshift-data:DescribeTable",
        "redshift-data:BatchExecuteStatement"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:redshift:<redshift_region>:<account_id>:dbname:<redshift_cluster_name>/*",
        "arn:aws:redshift:<redshift_region>:<account_id>:cluster:<redshift_cluster_name>"
      ]
    }
  ]
}
  • Give the policy a name, click Next and Create policy to attach it to the role.

​
Step 3: Associate the IAM Role with the Redshift cluster

Now you need to give the Confidence IAM role permissions to run load jobs.
  • Go to the Redshift cluster you want to use in the AWS console.
  • Go to the Properties tab, scroll down to the Cluster permissions, click Manage IAM roles and then Associate IAM role.
  • In the dialog that comes up, the role you created in the earlier step should show up. Select the role and click Associate IAM roles.

​
Step 4: Create the Redshift Database and Schema

  • Open the query editor for the Redshift cluster you are using.
  • Create a Confidence database to keep the data separate from the rest of your data. Then switch the query editor to use that database.
CREATE database confidence;
  • The Confidence database creates and stores a schema to contain the tables. Confidence creates a user corresponding to the IAM role, and grant access to write to this schema for the IAM role. You can copy the SQL with the information pre-filled by clicking the Copy SQL button.
CREATE SCHEMA confidence;
CREATE USER "IAMR:<role_name>" PASSWORD DISABLE;
GRANT ALL ON SCHEMA confidence TO "IAMR:<role_name>";
GRANT ALL ON ALL TABLES IN SCHEMA confidence TO "IAMR:<role_name>";
ALTER DEFAULT PRIVILEGES FOR USER "IAMR:<role_name>" IN SCHEMA confidence GRANT ALL ON TABLES TO "IAMR:<role_name>";
The role name here is not the ARN, just the name (role-name rather than arn:aws:iam::191394936087:role/role-name).
The CREATE USER command may fail with “user already exists” if someone has logged in with that user, if so ignore the error and continue.

​
Step 5a: Configure a Metrics Data Warehouse

  1. Go to the Confidence App.
  2. On the bottom of the left sidebar, select Admin > Connections > Metrics Data Warehouse.
  3. Select Redshift.
  4. Enter the details from the earlier setup steps.
  5. Click Save.

​
Step 5b: Configure a Flag Applied Connector

For Confidence to be able to store assignment data in Redshift, you need to set up a connector between Confidence and Redshift.
Assignment data is information on which users were assigned to which variants in the experiments you run. Assignment data goes into exposure calculations. Metrics use exposure to calculate results in your tests.
This connector is a “Flag Applied” connector. The connector is the part responsible for writing assignment to Redshift that Confidence Metrics can later read. To set it up:
  1. Go to the Confidence App.
  2. On the bottom of the left sidebar, select Admin > Connections > Flag Applied.
  3. Click Create
  4. Select Redshift as destination.
  5. Enter the details from the earlier setup steps.
  6. Click Save.
When you click save or have entered the required details, Confidence tries to connect to Redshift and load some sample data. If you have mis-configured anything, you see an error message.

​
Step 5c: Configure an Assignment Table

For Confidence to use the stored assignment table, you need to set up an assignment table that reads from the Redshift table. You first need to create an entity, which represents the thing you’re experimenting on, like your users. To do so, follow these steps:
1

Go to the Confidence App

2

Navigate to the Redshift connection

On the bottom of the left sidebar, select Admin > Connections > Flag Applied and select the Redshift connection you created.
3

Click Create in the Assignment table section

4

Create or select an entity

Create a new entity or select an existing entity. Entities are the things you’re experimenting on, like your users. Enter User and specify the data type of the identifier that identifies the entity. For example, if you have a UUID that identifies your users, your primary key type is a String.
5

Enter assignment table name

Enter a name for the assignment table, such as flag_applied. This name should typically match the name you used in step 5b. Confidence can then read assignments from the destination table of your flag assignments.
6

Click Create

🎉 Well done! You are all set up and ready to go.

​
What’s Next?

The next step is to create a fact table, and a metric. For an overview, see the metric introduction page, and the metrics quickstart.

Metrics Introduction

Overview of metrics in Confidence

Fact Tables

Configure fact tables for metrics

Assignment Tables

Set up assignment tracking

Configure Metric Quickstart

Create your first metric