AI proof of concept vs production AI system: why most demos never ship

What a proof of concept actually proves, and what it cannot

A PoC is a narrow experiment: a handful of representative inputs, a prompt or two, a model API and an output that looks right. It is worth doing: it settles whether the task is within reach of current models and whether the people who would use the output find it useful.

What it proves is feasibility on curated examples. What it cannot prove:

• Behaviour on the long tail: the inputs nobody thought to put in the demo set, and production sends them anyway.
• Consistency: an acceptable answer on the thousandth call as well as the first.
• Speed and cost at the volume the business actually runs.
• That the output can be acted on without a person checking it first.
• That anyone will notice when it quietly stops working.

The mistake is not building a PoC. It is treating a passed PoC as a nearly finished system and budgeting what remains as polish. The demo is the smallest component of the final system. Everything that makes the output dependable sits around it, and at the demo stage none of it exists.

The production gap, item by item

Each item below is an engineering problem with a known solution; the cost of shipping a PoC is roughly their sum.

Data: real volume, messy inputs, freshness

The demo ran on clean PDFs and well-formed records. Production receives scanned faxes, email threads quoted several replies deep, and fields that were mandatory on the form but empty in the database. Volume changes the problem too: an index that behaves on a few hundred documents behaves differently on a few hundred thousand, and the sources keep changing after it is built. Production needs ingestion that validates and normalises inputs, quarantines what it cannot process, and keeps the index current.

Evaluation: without a golden set, you cannot tell if it is getting worse

Most PoCs are judged by reading a few outputs and nodding. That is fine for a demo and fatal for a system. After any prompt edit, model upgrade or index rebuild, the only way to know whether quality moved is a fixed set of inputs with agreed answers, graded the same way every time. Without that set every change is a guess, regressions are found by customers, and eventually the team stops changing anything at all. Build the harness before any tuning.

Latency and cost at real load

A demo serves one user who is prepared to wait. Production serves many who are not. The largest model at every step, prompts stuffed with context and serial chains of calls are fine in a notebook and expensive in a queue. At real load the questions become which steps can use a smaller model, what can be cached, run in parallel or batched overnight, and what the bill per feature looks like when usage doubles. None of that is known until it is measured.

Security and tenancy

The PoC ran with one API key in a config file and one user who could see everything. Production has to answer who can see which documents, how one customer's data is kept apart from another's, what leaves your environment, how secrets are stored and rotated, and how the system resists an input that tells the model to ignore its instructions. Retrieval must respect permissions, not only relevance. Actions need an audit record, and the consequential ones need a person to approve them.

Integration with systems of record

In the demo the output was read on a screen. In production it has to land somewhere: a CRM, an accounting package, a ticketing system, a database other software depends on. That means idempotent writes so a retry does not raise a duplicate invoice, reconciliation so a missed webhook gets noticed, and a rule for when the downstream system is unavailable. Integration is easy to leave out of the estimate and impossible to leave out of the system.

Failure handling

Model APIs time out, rate-limit and occasionally return nonsense with complete confidence. A demo is re-run by hand. Production needs timeouts, backoff, dead-letter queues for work that cannot complete, fallbacks to another model or a simpler path, and defined behaviour for low-confidence outputs, usually a hand-off to a person. This is designed into the orchestration, not patched on afterwards.

Observability

When a production output is wrong, somebody has to be able to answer: what was the input, what did retrieval return, which prompt version and model produced this, how long did it take and what did it cost. That takes tracing across every step rather than console logs, plus quality metrics on a dashboard a non-engineer can read, with alarms on drift.

Operations and ownership

Someone has to run it. Deployments, model upgrades, prompt changes, index rebuilds, incident response and the monthly bill all need an owner, a runbook and a way to roll back. A PoC lives on a laptop or in a vendor sandbox. A production system lives in your own cloud account as infrastructure-as-code with CI/CD and documentation, so it survives the departure of the person who built it.

A production-readiness checklist

Audit a PoC against this table before committing budget to ship it. Anything still in the middle column is a work item, not a detail.

Structuring the path from PoC to production

Jumping from a passed demo straight to "build the whole thing" is how these projects run long. The path that works is staged and measurable; it borrows the exit-at-any-boundary logic of the engagement model in how we work: each step has to earn the next one.

Start with an evaluation baseline. Assemble the golden set and score the PoC before changing anything. Every later decision is measured against that number. Expect it to be lower than the demo suggested.

Gate each stage on evaluation, not opinion. Each stage clears a threshold before the next begins: retrieval recall, then generation quality, then integration correctness, then any autonomy. A missed gate is information, not failure; it says where the remaining budget should go.

Build in 12-week blocks. A block is long enough to take one capability to production-grade and short enough that the business can stop at its boundary if the numbers are not there. Working software ships every couple of weeks.

Ship behind flags. The system goes live to a small internal group first, then to a slice of real users, with the existing process still running alongside. Flags let you widen or narrow exposure without a deploy, and roll back a bad model upgrade the same way.

Measure against the baseline the whole way. Every release is compared with the pre-production score and with the manual process it replaces. If the system is meant to save time, measure the time; if it is meant to lift accuracy, measure it on the same cases a person handled. That is what turns "the demo worked" into "the system is worth what it costs".

The people problem: who owns it after launch

A PoC is often one person's work, and that person frequently sits outside engineering. When nobody else understands it, the project is one resignation away from stalling. A production AI system needs an owner with the authority to change prompts, approve model upgrades and pause the system; domain experts who keep the test cases current; and an engineer on call who can read the traces. Name those roles before launch or the system decays quietly: the index goes stale, the scored examples drift from reality, and the first sign of trouble is a customer complaint.

Ownership is also a build decision. A system built inside a vendor's platform is hard to move: the prompts, workflows and data sit inside their product, on their terms. One built in your own account, with code, prompts and evaluation sets in your own repositories, can be run by the people who built it or handed to an internal team. That is our default: the IP is yours from day one of any paid stage, handover is included, and taking the software in-house is always an option.