Nessoris
Book a call
← Insights

How do you take an AI pilot to production?

Take an AI pilot to production in 90 days: a 13-week plan with exit criteria for every stage, four baseline numbers and a five-check graduation gate.

Nessoris

Key takeaways

  • Pilots rarely fail at the demo. They fail at the handover, because nobody decided what production means, who signs for it, or what the pilot had to prove.
  • Write the exit criteria in week 2 and do not edit them afterwards.
  • A stage that misses its criteria repeats once, then stops. A stop at day 90 is a result: it costs 13 weeks, not a year.
  • Five checks gate graduation: decision owner, evidence of use, independent audit, rescue-or-rebuild call, run owner.
  • Redesign the workflow, not just the tooling: nearly three-quarters of AI high performers did (McKinsey, 2026).

Why do AI pilots stall before production?

More organisations are scaling AI, and earnings impact is not keeping pace. In McKinsey’s 2026 survey (1,719 respondents), 40% of respondents from organisations with revenue above US$1 billion report scaling AI agents, up from 27% a year earlier, while smaller organisations stayed flat at 22%. The share of “high performers”, those attributing at least 5% of EBIT to AI, stayed at about 6% (McKinsey, 2026).

  • 4 of 33 AI proofs of concept graduated to production, in IDC research for Lenovo (CIO.com, 2025).
  • 42% of companies abandoned most AI initiatives, up from 17% (S&P Global via CIO Dive, 2025).
  • About 6% of organisations are AI high performers, unchanged year on year (McKinsey, 2026).
  • More than 40% of agentic AI projects will be cancelled by the end of 2027, Gartner forecasts (forecast, 2025).

Three patterns recur, each with evidence behind it.

No baseline. A pilot that did not measure the old way cannot prove the new one is better. In a vendor-commissioned 2026 study of 397 decision-makers (FPT Software, with Forrester Consulting), 35% said they do not collect quantified metrics on AI and 10% do not measure AI outcomes at all.

No exit criteria. “Let’s see how it goes” produces a demo, not a decision. Gartner forecast in 2024 that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, and in 2025 that more than 40% of agentic AI projects would be cancelled by the end of 2027 because of “escalating costs, unclear business value or inadequate risk controls”. Both are forecasts, not measured rates.

The wrong kind of build. A pilot built to learn is fast, loose and disposable. A production system is none of those. When one quietly becomes the other, the organisation inherits prototype debt it never chose to take on. In LangChain’s agent-engineering survey of self-selected builders (1,340 responses), quality was the most cited barrier to production at about a third of respondents, ahead of latency at 20%. Data is the blocker in regulated sectors: 80% of banks in KPMG’s Q3 2026 pulse name data readiness or access as their top agent-deployment challenge.

All three have the same cure: write down, before the build, what must be true at each stage for the work to move on.

What does a 90-day AI pilot-to-production plan look like?

Ninety days is thirteen weeks. That is long enough to put one workflow through a learning build, a hardened production slice and a decision, and short enough that nobody forgets why it started. There is no published benchmark for how long a pilot should take, so this timeline is a design choice, not an industry median.

Weeks Stage What happens Exit criteria (all must be true)
1 to 2 Baseline Choose one workflow. Measure it as it runs today. Four baseline numbers recorded; a named decision owner; day-90 thresholds written and signed.
3 to 6 Learning build Run the workflow in a sandbox on masked or synthetic data. Grow the evaluation set from every correction. 20 to 50 evaluation tasks from real failures, with a written pass threshold met; supervision cap stated; one baseline number moved on the sample; the owner confirms the workflow is worth running.
7 to 10 Production slice A bounded slice runs on real data under production controls: independent audit, enforced limits, staged release, tested rollback. High-severity audit findings closed; rollback tested; change-failure rate no worse than baseline; cost per accepted change known; a run owner named with a service level and an on-call rota.
11 to 13 Graduation gate The slice runs on live work while the gate is prepared. Five gate checks pass, or the owner signs one extension or a stop; next-quarter scope agreed.

Three rules keep the plan honest. The exit criteria are written in week 2 and not edited afterwards. A stage that misses its criteria repeats once, then stops. A stop at day 90 is a result, not a failure.

Weeks 1 to 2: what to baseline before an AI pilot starts

Pick a workflow that is frequent, bounded, measurable and reversible: test generation for one service, a class of defect fixes, a document-processing queue. Avoid anything a regulator would call a critical function until the second quarter.

Then measure it as it runs today. For software delivery we record four numbers:

  1. Cycle time from request to merge.
  2. Change-failure rate: failed deployments divided by all deployments.
  3. Rework rate: unplanned deployments caused by production issues, divided by all deployments.
  4. Cost per accepted change: fully loaded delivery cost divided by changes merged after review and not reverted within 14 days.

For other workflows use the equivalents: time to complete, error rate, rework rate and cost per accepted output. Pull eight weeks of history, then watch the workflow run for two. Anthropic’s rollout guidance for its own coding agent is to start with a small pilot group and “establish a baseline before wider rollout” (Anthropic, 2026).

Weeks 3 to 6: why a learning build is not an open-ended pilot

A learning build answers one question: does this workflow work with AI, on this data, with this team? It is disposable by design. Two assets matter more than its output. The first is the harness: the conventions, decisions and tests the system reads before it acts, updated after every correction. The second is the evaluation set. Anthropic’s guidance is to prioritise volume over polish, to mirror the real task distribution including edge cases, and to grade with a different model than the one that produced the output (Anthropic documentation). Anthropic’s engineering team suggests starting with 20 to 50 tasks drawn from real failures (Anthropic, 2026); build yours from the corrections made in weeks 3 and 4.

By week 6 the decision owner should be able to say, from evidence, whether the workflow is worth running. If not, stop: four weeks is a small price for a clear no.

Weeks 7 to 10: how to harden a production slice

The slice is small on purpose: one team, one service or one class of change, on real data. Everything optional in the learning build becomes mandatory. An independent audit covers security, secrets, data model, tests, observability and running cost, read by someone who did not build it. Declared limits on what the system may do are enforced in tooling. Releases are staged, and rollback is tested before it is needed.

Expect pressure on stability. Google’s 2025 DORA report (about 5,000 respondents) found AI adoption now associated with higher throughput but also with increasing instability, which it attributes to higher change volume exposing weak testing and feedback loops (Google Cloud, 2025). Microsoft’s guidance splits evaluation the same way: pre-production evaluation and red teaming, then post-production monitoring of sampled live traffic with alerts on latency, errors, quality and token use (Microsoft, 2026).

Weeks 11 to 13: what the graduation gate checks

Nothing moves from learning to production status until five checks pass:

  1. Decision owner: a named person signs that the workflow should exist.
  2. Evidence of use: real work, and at least one baseline number moved.
  3. Independent audit: findings closed or accepted in writing.
  4. Rescue-or-rebuild call: which of the learning build’s code survives, and which is rewritten.
  5. Run owner: a named team with a service level, an error budget and an on-call rota before the first real customer record arrives.

Our rule of thumb for reading the gate: five of five, graduate and agree the next quarter’s scope. Three or four, the decision owner can sign one extension against the unmet checks, after which it is graduate or stop. Two or fewer, do not graduate: stop, or return to the learning build with a new set of written exit criteria.

Graduation should change the workflow, not bolt AI onto the old one. McKinsey found that nearly three-quarters of high performers fundamentally redesigned workflows, against about a quarter of other respondents (McKinsey, 2026).

Do regulations change the plan?

They shape the production slice. Vietnam’s Law on Artificial Intelligence (Law No. 134/2025/QH15) took effect on 1 March 2026 with three risk tiers, self-classification by providers, and human oversight, logging and incident management for high-risk systems. Existing systems have a transition period, to 1 March 2027 in most sectors and 1 September 2027 in finance, health and education (Rajah & Tann, 2026; Duane Morris, 2026; law-firm summaries, not legal advice). Check current dates for any other jurisdiction you deploy in, including the EU AI Act, whose high-risk timetable has been under revision.

Where we fit

Taking a pilot to production is one of the five scoped ways into our AI delivery work: an agreed scope, a written acceptance test and a named person accountable for the work. If the pilot is already live and struggling, the usual first step is an Express Audit: about a week, from US$2,500, fixed.

Sources

  1. McKinsey & Company (2026) The state of AI in 2026: On the road to ROI, 25 August 2026; n=1,719, 97 countries, fielded 4 May to 8 June 2026. mckinsey.com
  2. CIO.com (2025) “88% of AI pilots fail to reach production, but that’s not all on IT”, citing IDC research with Lenovo, 25 March 2025 (secondary; sample not disclosed in the article). cio.com
  3. CIO Dive (2025) on an S&P Global Market Intelligence survey of more than 1,000 respondents in North America and Europe, 14 March 2025 (secondary). ciodive.com
  4. FPT Software with Forrester Consulting (2026), global study of 397 decision-makers, 8 July 2026; vendor-commissioned (secondary report). itbrief.co.uk
  5. Gartner (2025) “Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027”, 25 June 2025 (forecast). gartner.com
  6. Gartner (2024) “Gartner predicts 30% of generative AI projects will be abandoned after proof of concept by end of 2025”, 29 July 2024 (forecast). gartner.com
  7. LangChain (2026) State of Agent Engineering; 1,340 responses fielded 18 November to 2 December 2025, self-selected. langchain.com
  8. KPMG (2026) Q3 AI Pulse, 24 September 2026; n=314 US leaders at organisations with revenue of US$1bn or more, banking supplement n=100. kpmg.com
  9. Anthropic (2026) “Demystifying evals for AI agents”, 9 January 2026 anthropic.com; and Claude Code documentation, “Manage costs effectively” code.claude.com.
  10. Anthropic documentation, “Define success criteria and develop test cases” (undated). platform.claude.com
  11. Google Cloud (2025) “Announcing the 2025 DORA report”, 23 September 2025 cloud.google.com; definitions of change-failure and rework rate per RedMonk (2025), 18 December 2025 redmonk.com.
  12. Microsoft (2026) “Observability in generative AI”, Microsoft Foundry documentation, updated 31 July 2026. learn.microsoft.com
  13. Rajah & Tann Asia (2026) “New Law on Artificial Intelligence takes effect on 1 March 2026”, 20 January 2026 rajahtannasia.com; Duane Morris Vietnam (2026), 3 March 2026 duanemorris.com.

Figures were checked against each publisher’s own page on 12 October 2026 unless marked as secondary. Vendor-commissioned and single-firm figures are labelled as such. This article is general information, not legal or investment advice.

AI deliverypilotsproduction readiness

Questions and answers.

Each answer stands on its own.

How long should an AI pilot run before production?

Long enough to hit its written exit criteria, and no longer. On a 13-week plan that is four weeks of learning build and four of production slice. If a workflow cannot show evidence in that time, more time rarely helps. No published benchmark sets a standard duration.

What is the difference between a pilot and a learning build?

A pilot is often judged by whether people liked it. A learning build is judged by exit criteria written before it starts, runs without production data, and has a date on which it is either promoted through the graduation gate or deleted.

What should we measure before starting an AI pilot?

Four numbers: time from request to completion, failure rate, rework rate and cost per accepted output. Pull eight weeks of history and observe the current process for two more weeks before the build starts.

Can we run several AI pilots in parallel?

Run one per team. Parallel pilots compete for the same decision owner, and the scarce input at the graduation gate is a signature, not engineering time.

What if the AI pilot fails its exit criteria?

Repeat the stage once with the same criteria, then stop. A stop at day 90 costs thirteen weeks, not a year, and the evidence from the learning build can be reused.

Orchestrate/Obsolete

Our thinking lab.

How the world of delivery is changing, and how a new kind of delivery company runs: partners, specialists and AI agents working as one system. New pieces by email when we publish.

One email when we publish, nothing else. Unsubscribe any time.