The Feature Flag Toolbox: Cloud, Edge, and Local

Feature flagging sits at the heart of modern experimentation, it is what enables teams to control user experiences remotely and run experiments directly from Confidence. But here's the thing: while feature flagging should feel simple and plug-and-play, building a resilient SDK that doesn't slow down your application is a challenge.

At Spotify, we've spent years navigating the tradeoffs between latency, resilience, and flexibility across different parts of our infrastructure. Some services can't tolerate the additional latency of a network call. Others need to target specific user cohorts that require lookups. Critical paths can't depend on external services being up.

This experience shaped how we built Confidence. Rather than forcing everyone into a one-size-fits-all approach, we support multiple resolution methods—from cloud resolving for simplicity, local resolution for performance, to edge deployment for serverless architectures. And we built it all on OpenFeature, so you're never locked into our platform.

In this post, we walk through these approaches, the tradeoffs involved, and how to choose what makes sense for your infrastructure. Our goal is simple: remove the technical complexity so you can focus on learning from users, de-risking launches, and making better product decisions.

How you resolve

When it comes to implementing flags in your application, it’s of course important that the feature flagging provider has good support for the runtime and language of your systems. Another major concern is vendor lock-in: Feature flags end up all throughout your codebase, making it risky to depend on a vendor without portability.

On the infrastructure side, we want to enable the commoditization of feature flagging and actively work against vendor lock-in. Instead of building a bespoke ecosystem, we decided early on to be fully compatible with the Cloud Native Computing Foundation (CNCF) open standard for feature flagging: OpenFeature.

OpenFeature offers SDKs in many languages for resolving feature flags with an industry-standard API. However, we noticed early on that mobile support was lacking, so the Confidence team built OpenFeature SDKs for iOS and Android and donated them to the open source community.

To use Confidence with OpenFeature, we created a Confidence plugin (called a "provider" in OpenFeature terminology) for each SDK to connect the OpenFeature API to the Confidence backends. This gives our users an OpenFeature-based integration layer that is both vendor-neutral and maintained by a large open source community.

Where you resolve

After you implement the feature flags in your code, it’s time to think about what technical requirements you have on the resolving itself. Key considerations include:

• Latency: In high performance services, a network call to evaluate a feature flag could be too slow.
• Resilience and third-party dependencies: For critical paths, relying on network calls to external services may not be acceptable.
• Cohort targeting: targeting a fixed cohort of users often requires look-up network calls.

At Spotify, we've navigated these tradeoffs across different parts of the business with high demands in every direction. This experience has shaped how we've built our SDKs and the multiple resolution methods we support.

Cloud resolution: network calls for each resolve

Cloud resolution has been the industry standard (and Spotify's approach) for a long time. It works by making a network call whenever a feature flag needs to be evaluated. Beyond being simple to set up and use, cloud resolution offers the most straightforward setup for features that require lookups.

For example, cohort targeting—the ability to target a predefined list of users—requires lookups against that list of users. With cloud resolving, Confidence can store these large cohorts in a BigTable instance and make a local network call from the resolver service when needed. This adds only a few milliseconds to the resolve request, meaning customers avoid the overhead of setting up and maintaining their own key-value store while keeping latency minimal. However, there will be network latency between the customer’s server and Confidence’s, which might or might not be acceptable in your application.

Client SDKs: cache flag values

Sometimes, feature flags are retrieved on the end-user device, such as a mobile app or client-side of a web page. The cloud approach would fetch flags one by one over the network whenever needed. We've optimized this by fetching all flags targeting the end-user at the start of the session and storing them locally. Our SDKs then read flag values from local storage throughout that session, eliminating repeated network calls. This works well for client applications where the user is the same for a longer time.

We call these Client SDKs, and they expose OpenFeature's "static context" APIs designed specifically for this scenario.

Local resolution: resolve in your application

As our user base grew, customers began asking for options with lower latency and more control over potential points of failure in parts of their systems.

For feature flagging in these systems, we designed a system that decouples flag resolution from our backend. The Confidence provider downloads flags and all the associated rules from experiments and feature gates. This state is updated periodically in the background. Once downloaded, all later resolutions are performed locally, in-process, taking only 10-50μs of pure computation with zero network dependency.

Beyond the performance boost, storing state locally eliminates downtime risk. In the unlikely event that Confidence goes down (our uptime is 99.99%), your flag evaluations continue resolving. Since flag rules and state are distributed via multiple CDNs, experiments using local resolution are completely unaffected by Confidence outages.

Edge resolution: resolve where you deploy

Local resolution is in-process resolution. The SDK runs directly alongside your service, downloading flag state at startup and then reusing it for many resolutions (while being periodically updated). This approach works well for long-running services, but creates challenges for serverless infrastructure. In serverless environments, each invocation would need to download the state separately, defeating the purpose of local resolving.

For this serverless architecture, we built edge resolvers. An edge resolver is a pre-packaged serverless function with all the flags and rules in memory, which means no download time at the startup of the function. These resolvers live close to your serverless application, delivering very low latency without the overhead of repeated state downloads. We make sure to deploy a new edge resolver for every change the customer makes in Confidence, so the feature flagging source of truth always matches with what the customer sees in our app.

Confidence currently supports edge resolving with Cloudflare and Fastly.

Summary

Feature flagging is just a means to experimentation, but its implementation details do matter. At Confidence, we've built a flexible system that adapts to your specific needs, whether that's the simplicity of cloud resolving, the performance of local resolution, or the edge deployment requirements of serverless architectures.

By building on OpenFeature, we've made it easy to get started while avoiding vendor lock-in. You get the power and reliability that comes from Spotify's experience running experiments at scale, with the freedom to resolve flags in the way that makes sense for your infrastructure.

The result is a feature flagging system that gets out of your way, letting you focus on what matters: building better products through experimentation.