97% accurate. Every prediction still checked by hand.

The classifier sorted inbound claim documents into fourteen routing categories. The model card said 97.2% accuracy. The business case had promised a 60% reduction in manual triage effort.

Actual reduction after fourteen months: zero. Not "less than hoped": the operations team opened, read and confirmed every prediction, exactly as they had before the model existed. The model had added a step.

The CTO's theory was that operations was resistant to change and needed a mandate from above. I spent the first day with the operations team instead.

The figure was honest, competently produced, and answered a question nobody needed answering. It was measured on a balanced test set: roughly equal numbers of each of the fourteen classes, assembled by the data science team so the model wouldn't "just learn the priors."

That is a reasonable thing to do while developing a model. It is a catastrophic thing to report to a business, because production does not look like that at all:

Three observations, in increasing order of how much they cost:

• The headline number is nearly all top-three performance. On the production distribution the naive baseline — always guess "routine correspondence" — scores 44%. Beating that with 97% is much less impressive than it sounds.
• Errors concentrate exactly where they hurt. The two worst-performing classes are the two the business cares about most. A misrouted litigation notice is a regulatory clock nobody started.
• Averaging hid it. Micro-averaged accuracy is dominated by high-volume classes. Nobody had ever looked at a per-class breakdown, because the single number was good and nobody asks a good number a follow-up question.

This was the part the CTO's "change resistance" theory got backwards. I asked the operations lead why she didn't trust the high-confidence predictions. Her answer took ten seconds:

"It says 0.94 on the ones it gets right and it says 0.94 on the ones it gets wrong. So what am I supposed to do with 0.94?"

She was describing a calibration failure, precisely, without the vocabulary. The model emitted confidence scores. Nobody had ever checked whether they meant anything. I plotted the reliability curve that afternoon:

Systematically overconfident, and — worse for her purposes — almost flat. Ninety-five percent of predictions sat above 0.90, and the difference in real accuracy between "0.91" and "0.995" was five points. The score carried almost no information about whether to trust an individual prediction, so the only rational policy was to check all of them. Operations was not resisting the model. Operations was responding correctly to a signal that was noise.

Report per-class, on the production distribution

The balanced test set stayed for development. Every business-facing number moved to production-weighted, per-class precision and recall, with the two critical classes reported separately and never averaged into anything.

Calibrate the confidence

Temperature scaling on a held-out set, fitted per class group rather than globally, because the rare classes were miscalibrated in a different direction from the common ones. The point is not the technique. It is that after fitting, 0.9 means 0.9, and the score spreads across the range instead of piling up above 0.90. Only then is a threshold meaningful.

Set the operating point by cost, not by F1

The team had been tuning for macro-F1, which treats a misrouted piece of routine correspondence as exactly as bad as a missed litigation notice. Operations knew the real ratio. Written down, it was roughly:

# cost of one misroute, in analyst-hours of downstream rework
routine_correspondence   0.2
standard_medical         0.5
proof_of_loss            1.5
litigation_notice       40.0   # missed regulatory clock
large_loss_fnol         60.0   # reserving delay, escalation

Thresholds were then chosen per class to minimise expected cost rather than to maximise a symmetric metric. In practice this means the common classes auto-route at a fairly relaxed threshold, and the two expensive classes route to a human unless the model is very sure, a policy any claims manager would have written by hand, and one no F1 optimisation would ever produce.

A second opinion where the errors are expensive

Calibration fixed the signal but did not fix the two classes that mattered. Litigation notices and large-loss first notices are rare, textually similar to routine documents, and the most expensive things in the estate to get wrong, so I stopped asking one model to carry them alone.

Anything the primary classifier places in a high-cost class, or that it places elsewhere with meaningful probability mass on a high-cost class, now goes to a second model from a different family for an independent read:

# high-cost classes only — 0.6% of volume, most of the risk
both agree, high confidence   → auto-route
both agree, low confidence    → review queue, ranked normally
models disagree               → review queue, top priority, both readings shown

The economics are not close. The council runs on well under one percent of volume, costs a few cents a day, and lifted large-loss recall further than any threshold change did, because the two models fail on different documents. Where a single model is confidently wrong, it is wrong alone; a disagreement is the cheapest possible early warning.

I deliberately did not extend this to the eleven ordinary classes. A council there would triple cost to protect against errors worth 0.2 analyst-hours. Multi-model review is a tool you point at expensive mistakes, not a default you apply everywhere.

Make the review queue a product

Instead of "review everything", the queue now shows what the model predicted, its calibrated confidence, the runner-up class, whether the second model agreed, and the reason it was queued. It is ordered by expected cost of error, so the most consequential item is always at the top. Reviewing a prediction became a five-second confirm-or-correct rather than a full re-read.

And every correction is written back as a labelled example. This is the part the original build discarded entirely: fourteen months of an operations team correcting a model, and not one of those corrections reached the training set. The humans were doing the model's hardest work for free and the model never learned anything from it. Closing that loop means the review queue shrinks over time instead of being a permanent tax, which is the difference between human-in-the-loop as a design and human-in-the-loop as a euphemism for unpaid labelling.

Note the last row. The number that had been reported for fourteen months went down. Recall on both classes the business actually cares about went up by twenty-five and twenty-nine points, and 58% of the manual effort disappeared. No model was retrained. Every gain came from measuring the right thing and choosing a threshold that reflected what errors actually cost.