The demo-to-production
gap in agentic UX

Every agentic product demo follows the same script. The user types a clear, well-formed request. The agent understands it on the first try, plans the right steps, executes them cleanly, and presents a complete result. Applause.

Production looks nothing like this. Real users phrase requests ambiguously. They change their mind halfway through. The agent gets three of five steps right and stalls on the fourth. The result is partial, and the user has to decide what to do with it. Interfaces optimized for the demo script tend to fail exactly here, often within the first weeks of real use, because they were never designed for the situation that actually dominates: imperfect input meeting imperfect execution.

This gap is not a model problem. It is a design problem, and it defines what agentic UX work actually is.

Why the demo script is so seductive

Demo-driven design is not new. We have always built happy paths first and error states last. But with conventional software, the happy path covers most real usage: a form either validates or it does not, a button either works or it does not. The unhappy paths are enumerable.

Agentic systems invert this ratio. The space of possible user intents is open-ended, the agent's interpretation of them is probabilistic, and execution runs across multiple steps, each of which can partially succeed. The "edge cases" are not edges anymore. They are the center of the distribution. A design process that treats ambiguity and partial results as polish-phase work has the priorities backwards.

There is also an incentive problem. Demos sell. Internally, the demo convinces stakeholders to fund the project; externally, it convinces customers to buy. Nobody funds a pitch that opens with "here is how gracefully we fail." So the failure states get designed by whoever is left in the room at the end, if at all.

What actually breaks in production

Three moments carry most of the damage.

The ambiguous request. A user writes "clean up the report" and means formatting; the agent deletes rows it considers redundant. The demo never shows this because demo requests are pre-tested. In production, the interface needs a way to surface the agent's interpretation before execution: what did it understand, what does it intend to do, in what order. This is the pattern usually called planning visibility, and current enterprise pattern work treats it as one of a handful of must-haves regardless of model or framework. It is not decoration. It is the only point where a misunderstanding is still cheap.

The partial result. The agent finished four of six steps, hit a permissions wall on the fifth, and skipped the sixth. What does the user see? Many interfaces show either a spinner or a final answer, with nothing in between. A partial result presented as a complete one destroys trust faster than an honest failure, because the user discovers the gap later, alone, without context. The interface has to make "here is what I did, here is what I could not do, here is why" a first-class screen, not an error toast.

The recovery moment. Something went wrong and the user wants to fix it. If the answer is "start over and phrase it better," you have converted an agent into an unpredictable command line. The systems that hold up in production make errors recoverable: a clear undo, easy correction of a single step, and a record of what happened that the user can actually read. Trust is rebuilt at exactly this moment or not at all.

Designing from the failure inward

The practical consequence: reverse the design order. Before designing what the agent looks like when it succeeds, design what it looks like when it half-succeeds.

A concrete way to start is to take the three moments above and treat them as required screens in every agentic flow, with the same status as the success state:

1. Interpretation screen. Show the agent's reading of the request and its intended steps before anything irreversible happens. Let the user edit the plan, not just approve or reject it.
2. Partial-result screen. Define upfront what the user sees when execution stops at step N of M. Which steps completed, which did not, what state their data is in now.
3. Correction screen. Define how a user fixes one wrong step without discarding the rest, and what the audit trail of that correction looks like.

If a flow cannot answer all three, it is not ready for production, however good the demo feels.

There is a second, quieter consequence for research. Testing agentic flows with pre-written, well-formed prompts reproduces the demo bias inside the lab. Usability tests need deliberately underspecified tasks, in the user's own words, so the interpretation and recovery screens get exercised as often as the success path. The interesting findings live where the request was ambiguous.

Takeaways

• The dominant real-world scenario for agentic products is imperfect input meeting imperfect execution. Design for that scenario first and treat the clean success path as the special case.
• Three screens deserve the same design investment as the success state: the agent's interpretation before execution, the honest partial result, and single-step correction with an audit trail.
• Test with ambiguous, user-phrased tasks. A study built on well-formed prompts is a demo with extra steps.

← All writing