Why AI Pilots Fail in Production: The Demo Had a Babysitter, the Rollout Did Not

What actually changes between the demo and the rollout?

Your pilot did not degrade. The supervision around it was removed. That is the entire failure, and it is why a second attempt with a stronger model usually fails in the same place.

Every stalled rollout I have been asked to look at follows one shape. The workflow still runs. The model still produces roughly the outputs it produced during the pilot. What changed is that the person who used to read the results, re-run the broken ones and quietly correct the exceptions has moved on to another project, and nobody replaced them, because nobody knew that job existed in the first place.

The measurement hides it. Pilots and rollouts are usually judged with the same number — accuracy on a sample — and that number was computed on outputs a human had already filtered. The filter was never a box on the system diagram. When the diagram became the production plan, the filter was not in the plan.

A pilot succeeds because a person is babysitting it and a rollout fails because nobody is, so what decides the outcome is not model quality but whether an operating loop — monitoring, escalation and a written owner — exists around the workflow.

What was the babysitter doing that nobody wrote down?

Five jobs, almost always the same five, and none of them appear in the pilot report.

None of this is exotic. It is the same runbook any manual process has, and it is skipped for AI workflows because the demo made the workflow look finished.

Why does a more capable model make this worse first?

Because it removes the loud failures. Production inputs are not the curated documents from the pilot; they are whatever arrives, including the 3am scan of a fax, a 90-page appendix, and a file with two tables in the wrong order. A better model handles more of them, so the obvious, embarrassing errors become rare — and the errors that generated human attention were exactly the obvious ones. What remains is a small set of failures that look correct on the surface: a plausible number attributed to the wrong clause, a summary that drops the one exclusion that mattered. Visible error volume falls, operator attention decays in proportion, and the residual failures become harder to catch than they were before.

Who owns the workflow at 2am on a Sunday?

If you cannot name one person, the workflow has no owner. Not a team, not the platform group, not the vendor. A name, in a job description, with hours in their week.

Three questions settle it, and I ask them before any rollout approval. Who gets paged when it fails? Who has the authority to switch it off, and does that authority require a meeting? And whose existing work gets deprioritised to answer the first two? An owner without allocated hours is an announcement, not a control. This is also the cheapest thing on the list to fix, which is why I find it strange how often it is the thing left until after the incident.

What does an operating loop contain, in parts you can specify?

Four parts: instrumentation, thresholds, escalation, fallback. Each is a written artifact, not a principle.

Instrumentation means every step writes a record: model version, prompt version, input identifier, output, latency, cost, and the decision taken. The version field is not optional. Without it you cannot distinguish a system that drifted from a system you changed on Tuesday. There is a longer treatment of how to design an AI workflow where a failed step is visible before a customer is, and it is worth giving to whoever builds this, because the design decisions happen at build time and are expensive to retrofit.

Thresholds should be three, not thirty. One on output shape: empty, truncated, schema-invalid. One on distribution shift: input categories you have never seen, or a rising share of runs taking the fallback path. One on rate: cost or volume per hour moving outside a band you set while the workflow was healthy. Every additional alert you add beyond these three lowers the chance anyone reads any of them.

Escalation is a named second person and a clock. "Someone will look at it" is not escalation, and neither is a Slack channel. The clock matters more than the response: knowing that a wrong output will be reviewed within two hours is what allows the fallback path to be short instead of indefinite.

Fallback is the manual process, still staffed at a reduced level, and exercised on a schedule. If the fallback has not been run in three months it is not a fallback, it is a paragraph. I have watched a team discover this during a provider outage, at a point when the people who knew the manual process had been reassigned two quarters earlier.

What does the loop cost, and where do those numbers come from?

Start with the industry prior, clearly labelled as a forecast rather than a measurement. Gartner's July 2024 press release predicted that at least 30 percent of generative AI projects would be abandoned after proof of concept by the end of 2025. The word doing the work there is "abandoned", not "failed" — projects that technically worked and were dropped because nobody carried them. That is precisely the gap this article describes, and it is a forecast, so treat it as a reason to check your own numbers rather than as evidence about yours.

Then do the arithmetic that is genuinely yours. A 40-page document runs roughly 20,000 words, which is on the order of 27,000 tokens depending on how tables and headers tokenise. Run 4,000 documents a month and you are at about 108 million input tokens monthly, plus output. That number is knowable in advance, and it is usually the only number anyone models carefully. The number that matters more: at 4,000 runs a month, a 0.5 percent exception rate is 20 exceptions a month, roughly one per working day. In a pilot that was one person's afternoon. In production it is nobody's inbox, and it stays invisible until an audit or a customer finds the pattern.

For supervision cost, I can only report what I see in the workflows I run and hand over: about one to two hours a week per workflow once it is stable, plus one dedicated person for the first month after rollout. Call it 0.1 to 0.25 of a full-time role, and budget it before the volume justifies it rather than after. That is the real decision on the table: you staff the loop at pilot volumes, or you keep the workflow in supervised mode, or you do not ship it. All three are defensible. Shipping unsupervised while telling yourself the model will handle the tail is the only option that reliably loses money twice.

What should be true before you approve a rollout?

Four gates, all cheap, none technical. A named owner with hours, confirmed by their manager. An escalation path with a second name and a response clock. A fallback that has been exercised within the last month, with the result written down. And one operational metric: human-touched runs per 1,000. That last one is the honest measure of automation, and it should trend down. If it stays flat, you built an AI-assisted manual process and should price it as one.

I would also put the supervision hours into the statement of work or the internal budget line, in the same document as the licence or API cost. Costs that live only in a meeting get removed in the next quarter's planning round, and this one gets removed first because nobody can point at what it produces. A written line item is a control, and it survives the departure of the person who understood why it was there.

What I would do differently on the next rollout

Write the operating loop during the pilot, not after it, because the pilot is the only period when someone is actually performing the five jobs above and can describe them accurately. Interview that person before they are reassigned, and treat their notebook as the specification. Then decide explicitly whether the workflow ships supervised or unsupervised, and put that decision in writing with a name next to it. The model will keep improving on its own schedule, which means the model is the one part of this system you do not need to manage — and the part you skipped is the part that decides whether the work survives contact with a full month of real volume.