August 10, 2026

AI Pilot Fatigue: Eight Steps From Pilot to Production

Why AI pilots stall before production — and the eight-step governance framework we use to ship AI into mortgage and insurance, where output is regulated.

Diagram of eight steps from AI pilot to production, grouped into four stages: Scope, Govern, Ground, and Own the Output

Enterprises are not tired of AI. They are tired of AI pilots that never reach production. Jason Salzetti, chair and CEO of Deloitte Consulting, calls it “pilot fatigue”: companies keep launching AI pilots and, by Deloitte’s own research, scale fewer than 30 percent of them. His diagnosis matches what we see in every regulated buyer we talk to: the limiting factor is rarely the model. It is the foundation around it — data, governance, and process redesign. Reading it prompted us to write down the framework we actually deliver against.

Here is the uncomfortable part: most stalled pilots worked. The model performed. The demo impressed. What failed was everything around it — nobody could say who signs off on the output, the data had no owner, the process it was meant to replace never changed, and when the security review or the auditor showed up, there was no evidence to hand over. Pilot fatigue is not an AI problem. It is a governance problem.

Salzetti’s seven principles are good ones: start with the work, let data guide decisions, establish governance early, listen to the people closest to the work. They get you seven-eighths of the way. Below is our field restatement — the steps we hold every engagement to — plus the step we have never seen on anyone’s checklist. In mortgage and insurance, that step decides whether the pilot ever leaves the lab.

The Framework: Eight Steps, Four Stages

Steps 1–7 decide what the AI is for. Step 8 decides what it says. Both have to be written down before the build starts, and both have to be owned by someone whose name is on the document.

Stage 1 — Scope: Pick the Number, Prove the Data

Step 1: Name the number you are moving. Not “where can we use AI?” — that question generates pilots, not outcomes. Start with the metric: cycle time on a file, cost per processed claim, touches per application. Then ask whether AI is even the right lever. Sometimes the honest answer is a workflow fix and no model at all, and a team that can tell you that is worth more than one that can’t.

Step 2: Prove the data exists — and has an owner. Before anything is greenlit, confirm the data the use case depends on is real, clean enough to use, and owned by a named person who can answer for it. If the inputs live across three systems with no reconciliation, that integration is the actual project. Most stalled pilots die here, quietly, a quarter before anyone notices.

Stage 2 — Govern: Sign-Off and Scope, Before a Line of Code

Step 3: Decide who signs off before you build. Write down who approves what the AI touches and who is accountable when it is wrong. Do this before the pilot, not after it works. Retrofitting accountability onto a working demo is how a successful pilot sits frozen for six months — everyone agrees it works, and nobody is willing to own it.

Step 4: State the scope in one sentence. “We use AI to cut manual effort in document intake — nothing borrower-facing yet.” Every proposed pilot either fits the sentence or waits. This is the cheapest governance control that exists, and it is the difference between a portfolio and a pile.

Stage 3 — Ground: Real Workflow, Real Operators, Real Process Change

Step 5: Ask the people doing the work. Not a survey — a direct conversation with whoever is re-keying data between systems or answering the same borrower question forty times a week. They already know where the time goes. Their answer is the requirements document.

Step 6: Build for one workflow, not everyone. A generic assistant solves nobody’s specific problem well. The pilots that scale are narrow: one workflow, one team, one measurable number from Step 1.

Step 7: Change the process with the tool. New AI dropped onto an unchanged workflow does not stick — the team routes around it within a month. The tool, the process, and the team’s daily routine have to move together, which is why this step is organizational work, not engineering work, and why it is the one most often skipped.

Stage 4 — Own the Output: The Step Every Checklist Misses

Everything above governs what the AI is for. None of it governs what the AI actually says — and in a regulated business, that is the only part your customer, your regulator, or your auditor ever sees.

Step 8: Business authors the response, scenario by scenario. The business owners — product, operations, compliance — specify, in writing, how the system behaves in every exchange it is allowed to have. What gets summarized, and how deeply. What the system refuses. When it escalates to a human. What it must never state as fact. How the response reads to the person receiving it — because it goes out under your name, not the model’s.

Those are the headline categories. The full scenario inventory runs considerably deeper. Building it — pulling those decisions out of the people who own them and turning them into something engineering can enforce — is half the work of getting an AI system past a regulator.

This is not a technical specification, and it cannot be delegated to the engineering team, because these are not engineering decisions. An engineer can implement a refusal; only the business can decide what gets refused. If nobody writes this document, the model writes it by default — one improvised response at a time, in whatever voice it generates that day, on your letterhead.

In Mortgage and Insurance, the Response Is a Regulated Act

In most industries, Step 8 is good product hygiene. In lending and insurance, it is compliance.

When a system summarizes a loan file for an underwriter, the choice of what to omit is not a design preference — it shapes a credit decision that fair-lending law scrutinizes. If AI-assisted output feeds into a denial, the reasons given to the borrower are governed by adverse-action requirements under the Equal Credit Opportunity Act. The summary is not a convenience layer. It is part of the decisioning record.

Insurance is no different. Claims correspondence — what a carrier tells a claimant about the status and disposition of a claim — is regulated under state unfair claims settlement practices laws, and it is exactly what market conduct examiners read. An AI system that drafts or condenses claims communication is producing regulated correspondence, and “the model decided what to include” is not an answer any carrier can give an examiner.

This is why the eight steps end where they do. In these verticals, the response specification is not an enhancement to the pilot — it is the artifact that determines whether the pilot can legally become a product.

Enforce It as Code, Not as a Prompt

A response specification that lives in a system prompt is a suggestion. Models drift, prompts get edited, and “usually behaves” is not a control any auditor will accept.

Our rule on the Reslt Platform is that what runs in production is deterministic code — never a model improvising in front of a customer — with the audit evidence generated alongside the software rather than assembled under deadline when the security review lands. How a business-authored specification becomes deterministic, evidence-producing software is the core of how we deliver, and it is the part we don’t write blog posts about. What we can say is that this posture took our work through SOC 2 Type II with zero findings and past Fortune 100 security reviews — not because the models were clever, but because every output the system could produce had an owner before it produced one.

The Question That Decides Whether Your Pilot Scales

Scaling is not a bigger pilot. It is the governance, evidence, and ownership work that nobody put into the pilot — done late, under pressure, if it gets done at all. The eight steps are simply that work, done early and in order.

So the question worth asking about any AI initiative on your roadmap: who owns the document that says what the AI is allowed to say — product, compliance, or nobody? If the answer is “nobody,” that is the project. Everything else is a demo.

If you are staring at a pilot that works but will not survive a security review, that gap is precisely what our four-week Enterprise-Readiness Engagement exists to close.

Related reading: Compliance-First LLM Integration for Mortgage & Insurance · Mortgage Automation & Digital Origination in 2026

Facing the same problem?

A short conversation tells you exactly where you stand with enterprise buyers.

Book a readiness call