The Expensive AI Failure Is the System You Kept, Not the Pilot You Cancelled

Which failure actually costs money?

Two things go wrong in enterprise AI, and only one of them appears in a project report. The pilot cancelled at week ten costs a bounded amount of money and ends on a date you chose. The system that ships at the wrong scope, gets adopted quietly, and becomes load-bearing before anyone re-examines the design costs several times more — and it does so while everyone involved believes it is working.

The expensive AI failure is not the abandoned pilot, it is the wrongly-scoped production system that becomes load-bearing before anyone notices the design was wrong.

A cancelled pilot has one cost and one uncomfortable meeting. A production system has a build cost, a maintenance cost, an integration surface, and the part nobody prices in advance: a dependency graph that grows around it every month it runs.

What does a failed pilot actually cost?

Be specific, because the number is smaller than the fear. Take a six-week pilot with one engineer and a part-time operations lead. Two hundred hours of engineering time at a loaded rate of roughly $90 an hour is about $18,000. Sixty hours of operations time at $65 is $3,900. Inference for a few thousand test runs, call it $200. A vendor evaluation that consumed three meetings and two afternoons, perhaps $2,000. That is roughly $25,000, and it is finished on a date you set in advance.

The loaded-rate multipliers behind those two figures are conventions rather than measured benchmarks: multiply base salary by 1.25 to 1.4 to cover employer taxes, benefits and overhead, divide by paid hours, then adjust downward for meeting load. Use your finance team's own multiplier. What matters here is the shape, not the precision — a pilot is a bounded number with an end date and a decision attached to it.

What you own at the end is real, and it is not obtainable any other way at that price: the knowledge that this step is harder than it looked, which step surprised you, and which input was the problem. The pilot that fails honestly is the cheapest research available. That is why "we tried AI and it did not work" is such a poor reason to stop.

What makes a wrongly-scoped system expensive?

Not the build. The dependencies.

A system becomes load-bearing when other work starts to assume it, and that happens in a sequence where every step is a reasonable local decision and none of them looks like a commitment.

Someone stops keeping their own copy. The duplication was manual and annoying, so removing it looked like tidying up. It was actually the last place the information existed independently.

A downstream report starts reading from it. A dashboard, a forecast, a compliance summary. Nobody decides to depend on the system; the fastest way to get the report done is to point it at the new output.

A second team plans around it. Their timeline now contains an assumption about your release schedule, which they have not written down anywhere and will not remember making.

The exception queue becomes someone's job. A process that produced occasional exceptions in a pilot produces a permanent queue in production, and the queue needs an owner with judgement. That owner is now the person who cannot let the system change shape.

Each of these converts a technical decision into an organisational one. The technical reversal stays cheap for a long time — re-map the fields, re-scope the unit, re-prompt, re-test. The organisational reversal is not cheap, because it means telling four people that something they now depend on has to change, and paying for parallel running while it does.

Where does the reversal cost actually land?

A worked example, with the assumptions stated so you can move them.

A claims-intake step runs 40,000 items a month. The build automates field extraction and commits the fields, with human review on flagged items. That was the wrong unit: the expensive part of this process was never the typing, it was the routing decision about which queue an item belongs in. But extraction is easier to demo, so extraction is what got scoped.

The consequences are arithmetic. With the wrong design, roughly 18 per cent of items need human attention — call it 7,200 items at six minutes each, 720 hours a month, about $46,800 at a loaded $65 an hour. A correctly scoped version, where the model proposes a route and a human confirms, holds exceptions near 5 per cent: 2,000 items, 200 hours, about $13,000. The gap is $33,800 a month, and it recurs until somebody notices, which in my experience takes about a year because every individual month looks survivable.

Then the fix. Eight weeks of parallel running while the new scope is validated, an engineering cost in the low tens of thousands, plus the reshaped queue, plus the four downstream conversations. Nothing about that sequence is dramatic. It is just expensive in a way that never appears as a single line item.

Why does the pilot not catch the wrong scope?

Because a pilot and a production system answer different questions. A pilot validates capability: can the model do this, on clean samples, with someone watching closely. Production validates fit: is this the right step to hand over, at this granularity, under these exception rules, owned by this team. Capability is measurable with two hundred examples. Fit is measurable only with volume, and volume is the thing a pilot deliberately does not have.

The scoping error is usually a unit error — automating the step above or below the one that actually costs money. I have watched a document-extraction build hit 96 per cent field accuracy and still lose money, because the judgement that mattered was which queue to route to, and extraction simply moved that judgement downstream to a place where it was more expensive to make. The model worked. The scope was wrong. Those are separate failures, and only the first one shows up in an accuracy report.

Which signals say a system is becoming load-bearing?

Most of them are visible months before anyone calls it a problem. If two or more are true and the design has not been reviewed since the pilot, you are not deciding whether to re-scope. You are deciding how much the re-scope will cost.

What does a slower, smaller first step actually look like?

Slower and smaller are not the same as vague and prolonged. The purpose of a smaller first step is to keep the scope decision reversible for longer, and three constraints do most of that work.

One decision, not one process. Scope the automated unit to a single decision with a named owner. When the design turns out to be wrong, you change the unit without touching the surrounding workflow.

A dependency budget. Before the build starts, write down how many downstream things are permitted to read from it. Zero for the first quarter is a legitimate answer, and usually the correct one.

A pre-committed stop condition. Written before the build, it names the exception rate or cost per item at which you will reduce scope instead of pushing through. This is the part a 90-day roadmap that names one workflow and pre-commits its stop condition gets right: it puts the scope decision on a schedule rather than leaving it to be discovered in production a year later.

The honest counter-argument matters too. Sometimes pushing through is correct. If a system is working, adopted, and the design flaw costs money once rather than monthly, tearing it down to chase a cleaner architecture is the sunk-cost fallacy running in reverse — you would be paying real money for tidiness. The test is whether the error recurs. A recurring error is arithmetic and migration wins. A one-time awkwardness is not worth a migration.

Why slow down at the scope decision rather than at the start?

The scope decision is the one genuinely irreversible choice in the sequence, and it is usually made in a hurry by whoever is closest to the demo. A slower first step is not smaller ambition; it is a commitment kept cheap to undo until evidence exists, which is the opposite of what most approval processes optimise for. So the question worth asking is not how long the pilot will take, but what happens if the pilot succeeds at the wrong scope — cheap to ask in week one and extremely expensive to ask in month fourteen. Put your caution on the scope decision and the dependency budget, write both down before the build starts, and cancel a pilot without regret when it deserves it; the cost of a bad AI decision is rarely the money spent learning something, it is the money spent every month on a design nobody revisited.