Confidence Loop: Building products that continuously improve

Building a great product isn't only about deciding what to build next.

A large part of product development is much less visible: understanding where users struggle, fixing small bugs, smoothing out rough edges, keeping documentation accurate, tuning existing experiences, and making sure the product actually behaves the way we intended.

We think agents give us an opportunity to approach this work differently.

We're building Confidence Loop: a system that turns signals from real users into product understanding and improvements and, over time, allows more and more of those improvements to happen automatically.

#Users are already telling us what to improve

Users are constantly producing signals about the product, often without explicitly giving us feedback.

A failed action is a signal. An error log is a signal. So is a sales call, a recording of someone using the product, an analytics event, a support conversation, or a feature request.

Individually, these signals can be noisy. Collectively, they contain a rich picture of what users need, where they struggle, and where the product falls short.

Most of this information is scattered, unstructured, and difficult to act on.

Confidence Loop continuously turns those signals into product understanding and eventually into improvements.

#Two kinds of product work

The more interesting idea behind Confidence Loop is how agents might change the way we divide product work.

We've started thinking about product development as having two modes: high-intent product work and low-intent product work.

High-intent product work starts with an idea or a hypothesis.

You've identified an opportunity to make the product meaningfully better for users, and you have some intention for how the product should change.

You explore the problem, bring together what you've learned, develop a point of view, design a solution, and build it. Increasingly, you might do all of this together with an agent.

The direction comes from you. You're deciding what the product should become.

Then there's low-intent product work.

Low-intent doesn't mean low-value. It's the continuous stream of smaller improvements that comes from putting a product in front of real users.

A workflow is more confusing than it needs to be. An error keeps showing up. Documentation is out of date. Users repeatedly ask for the same small capability. A feature works, but there are paper cuts around it. Something behaves differently from what users expect.

No one necessarily needs to wake up with an idea to fix these things.

The opportunity is already there, encoded in the signals users are producing.

The work is to notice it.

Today, that still requires a surprising amount of human attention. Someone has to encounter the signal, recognize that it matters, connect it to other signals, understand the underlying problem, decide what should change, and then make the change.

Because attention is limited, many of these opportunities simply don't get addressed.

We think agents can change how products are built.

If high-intent work starts with what we want the product to become, low-intent work starts with what the product is telling us needs to get better.

Confidence Loop makes the product increasingly capable of acting on that second category.

#How Confidence Loop works

Confidence Loop works in a few distinct stages.

It starts with signals. These can come from almost anywhere users interact with the product or tell us about their needs: product events, error logs, recordings, sales calls, support conversations, and more.

We analyze those signals and turn them into observations.

An observation is a concrete piece of product understanding derived from one or more signals. It might be a feature request, a bug, a usability problem, a pain point, or simply something unexpected about how people are using the product.

Observations are useful individually, but they become much more interesting when we can connect them.

Confidence Loop continuously looks across observations to identify patterns and clusters related observations into issues. Ten seemingly independent signals might actually be different manifestations of the same underlying problem.

This is where we move from what happened to what should we do about it.

For each issue, Confidence Loop can start an investigation. The investigation pulls together the evidence behind the issue, looks at the relevant parts of the product, and tries to understand the underlying cause.

From there, it can propose an appropriate next step.

Sometimes that's a code change. Sometimes it's a product or design change. Sometimes the right answer is better documentation, a mitigation, or simply surfacing the issue to a product builder because it points to a larger opportunity.

Every issue retains the evidence that led to it.

That gives us a chain from the proposed improvement all the way back to what users actually experienced:

Signal → Observation → Issue → Investigation → Improvement

But we don't think every step of that chain should always happen autonomously.

#Human judgment is part of the loop

The goal of Confidence Loop is to make better use of human judgment, not remove it from product development.

There are plenty of improvements where the right thing to do is clear. But product development is full of ambiguity. There might be several reasonable solutions. A seemingly small change might have consequences elsewhere. Or an issue might reveal a bigger product question that requires context, taste, or a decision about what we actually want the product to be.

When Confidence Loop isn't confident about which direction to take, it can ask a human.

The human doesn't have to manually discover the problem and assemble all the evidence first. Confidence Loop can bring the problem, context, investigation, and possible paths forward, then ask for judgment at the point where judgment is actually needed.

And different teams may want different levels of control.

Confidence Loop can operate at different levels of autonomy. At one end, it can simply collect signals and surface issues for humans to work on. Give it more autonomy and it can investigate those issues and propose solutions, while leaving implementation to the team. Go further and it can investigate and build fixes, with a human still reviewing or approving what happens. And for the kinds of work where you have enough confidence, it can close the loop autonomously.

The question is where human judgment is valuable, and where the loop has enough confidence to keep going on its own.

And that level doesn't have to stay fixed forever. As we understand which kinds of work Confidence Loop handles reliably, we can give it more autonomy in those areas while keeping humans closely involved in others.

#Better inputs for high-intent work

The first role of Confidence Loop is to make high-intent product work better.

Today, the evidence we need to make product decisions is often scattered across many different places: analytics, support conversations, sales calls, recordings, error logs, feature requests, and the accumulated knowledge of the people closest to users.

Connecting all of that evidence takes work.

Confidence Loop continuously turns those raw signals into structured product understanding.

That means when someone starts exploring a problem or developing a new idea, they don't have to begin from scratch. The relevant observations, issues, and evidence can already be there.

The loop becomes a continuously updated understanding of how the product is meeting reality.

Humans can use that understanding to make better high-intent decisions about where the product should go next.

#Automating low-intent work

The second role is more ambitious.

We want to automate increasingly large parts of low-intent product work.

If a system can continuously observe what is happening, identify recurring problems, group the evidence into issues, investigate their causes, and propose solutions, then an improvement no longer has to begin with a person noticing something and deciding to investigate it.

How far the loop continues depends on the problem, the confidence of the system, and the level of autonomy we've chosen to give it.

Sometimes the right outcome is an issue waiting for a human. Sometimes it's an investigation and a proposed solution. Sometimes it's a completed change waiting for review.

And eventually, for sufficiently safe and well-understood kinds of work, the entire loop can happen autonomously:

Signal → Problem → Fix → Evaluation → New signal

The product begins to participate in its own improvement.

#From episodic to continuous improvement

Most products today improve episodically.

People decide what to work on. They investigate. They ship something. They move on to the next thing.

Meanwhile, users continue producing an enormous amount of evidence about everything that could be better.

Some of that evidence eventually makes its way back into product development. Much of it doesn't.

Agents make it possible to imagine a different model.

Instead of relying entirely on humans to notice, interpret, and act on every opportunity, we can build systems that continuously close more of those loops themselves and know when they need human judgment to keep going.

That doesn't make intentional product development less important.

Quite the opposite.

There will always be questions that require taste, ambition, judgment, and a point of view about what should exist. That's high-intent product work, and it's where we want product builders to spend more of their attention.

But underneath that work, we imagine another process running continuously: observing how the product meets reality, finding the gaps, closing the ones it can, and bringing humans into the ones that require judgment.

Humans decide what the product should become.

Confidence Loop helps continuously understand what needs to get better and increasingly takes care of the work of making it better.

#Request a demo

Confidence Loop is now in early preview with design partners. Leave your email and we'll get in touch to set up a demo, and possibly make you a design partner.