Cursor builds a working AI agent in an afternoon. Turning that into a production AI agent (one running under real users in a regulated or multi-tenant environment) takes 3 to 6 months. The gap is 5 layers: eval harness, observability, error handling, audit trails, and compliance integration. This post walks through each one. What it does, what it costs to skip, and which one to build first.
A production AI agent is an autonomous agent that runs reliably under real users in a regulated, multi-tenant, or otherwise high-stakes environment. Not a demo loop, not an internal experiment, not a prototype with one happy path. Three markers separate it from a prototype: an eval harness that catches regressions on every commit, an audit trail that lets you reconstruct any decision the agent made, and error-handling that keeps a model outage from cascading into a customer-facing incident. The 5 layers below (eval harness, observability, error handling, audit trails, and compliance integration) are the architecture that turns a Cursor-built prototype into a production AI agent. The rest of this post walks through each layer.
A demo agent is a single happy path. A production agent is the same happy path plus everything that breaks around it. LangChain’s team frames the work as a new discipline: “agent engineering is the iterative process of refining non-deterministic LLM systems into reliable production experiences,” and the architecture you pick depends on how much workflow versus agency the system needs. (LangChain, Agent Engineering: A New Discipline, 2025)
The demo-to-production gap isn’t a model problem. GPT-5.5, Claude Opus 4.7, and Gemini 3.1 Pro are all production-capable on the inference axis. The gap is engineering: the layers around the model that decide whether a hallucinated tool call costs you a 4-line log entry or a 4-figure incident.
We have rebuilt agent prototypes for six different teams over the last 18 months. Patient intake at a digital health company, claims triage at a consumer-finance lender, and a dispatch assistant for a multi-trade FieldOps operator. Same pattern every time. The prototype works. The production version requires the 5 layers below.
The team building those rebuilds is 80+ professionals across Salt Lake City and Tirana (engineers, QA, DevOps, designers, PMs). Two timezones. Mid-market budgets. We have shipped under HIPAA for HealthTech clients and under SOC 2 for financial-services clients. We are not theorists.
An eval harness is an automated test suite that runs realistic agent inputs against ground-truth outputs on every commit. Without one, you ship blind. Every prompt change is a coin flip on whether you regressed last week’s bug fix.
The minimum viable eval harness is a dataset of 5–10 examples per agent capability, plus an evaluator that runs the agent against each example and scores the output. LangSmith’s documentation recommends starting exactly there: “5–10 examples of what ‘good’ looks like for each critical component,” focusing on agent behaviors like tool calls and argument formatting. (LangChain, LangSmith evaluation concepts)
A real eval test case for a patient-intake agent looks like this:
from langsmith import Client
from jsonschema import validate, ValidationError
client = Client()
dataset = client.create_dataset(
dataset_name="intake_agent_v1",
description="Schema-conformance + extraction accuracy for intake.",
)
client.create_examples(
dataset_id=dataset.id,
examples=[
{
"inputs": {
"transcript": "Pt is 47 yr old female. Allergies: PCN, sulfa. "
"On metformin 500mg BID. Reason for visit: f/u DM.",
},
"outputs": {
"age": 47,
"sex": "F",
"allergies": ["penicillin", "sulfa"],
"medications": [{"name": "metformin", "dose": "500mg", "freq": "BID"}],
"chief_complaint": "follow-up diabetes",
},
},
# 9 more like this, including adversarial cases
],
)
def schema_conformance(inputs: dict, outputs: dict, reference_outputs: dict):
"""Did the agent return JSON that matches the chart schema?"""
try:
validate(outputs, schema=CHART_SCHEMA)
return {"key": "schema_valid", "score": 1}
except ValidationError as e:
return {"key": "schema_valid", "score": 0, "comment": str(e)}
def allergy_recall(inputs: dict, outputs: dict, reference_outputs: dict):
"""Did the agent capture every allergy in the source?"""
expected = set(reference_outputs["allergies"])
actual = set(outputs.get("allergies", []))
return {
"key": "allergy_recall",
"score": len(expected & actual) / len(expected) if expected else 1.0,
}
client.evaluate(
intake_agent,
data="intake_agent_v1",
evaluators=[schema_conformance, allergy_recall],
experiment_prefix="intake-v1",
max_concurrency=4,
)
Sixty lines. Runs in CI. Catches the Sonnet-to-Haiku swap that quietly tanks recall by 12%.
Cost to skip: every prompt change is a hope. The first time a regulator asks “how do you know your agent didn’t degrade after you fine-tuned?” you have no answer. Anthropic’s production guidance recommends “extensive testing in sandboxed environments, along with the appropriate guardrails” before deployment, and eval harness is the harness for that. (Anthropic, Building Effective Agents, 2024)
Observability for an agent is structured logging on every model call, every tool invocation, every retry, every fallback. Not OpenTelemetry-as-aesthetic. Actual decision-tree visibility, so when the agent does something unexpected at 2:14 AM you can see why.
A production agent log entry should look like this:
{
"ts": "2026-04-27T14:32:08.412Z",
"request_id": "req_2K8bZx4n",
"tenant_id": "acme-clinic",
"agent": "intake_v3",
"step": "tool_call",
"tool_name": "lookup_patient",
"tool_args": {"mrn": "REDACTED"},
"model": "claude-opus-4-7",
"input_tokens": 1842,
"output_tokens": 211,
"latency_ms": 1217,
"retries": 0,
"fallback_used": false,
"decision": "lookup_patient(mrn=REDACTED)",
"downstream_status": "ok"
}
That shape gives you six things you cannot get from naked LLM logs: per-tenant attribution, retry count, fallback status, model identity, token cost per step, and the actual decision the agent made versus the input it saw.
We pipe these to a structured-log destination (Datadog, Honeycomb, or a self-hosted ELK stack, depending on the client’s stack), then build per-tenant dashboards on top. For a regulated client we add a separate immutable audit-log path (Layer 4). For SaaS vendors selling into low-tech operators (one of our four ICP verticals; see our custom software development practice for the broader engagement context), the per-tenant attribution is what lets you respond to “my dispatcher saw something weird at 2pm” with a 90-second answer instead of a 3-hour investigation.
Without observability, the post-incident retro is a guess. We have watched a team rebuild a feature flag from scratch because nobody could prove the old version had ever worked. Every CTO who has lost an afternoon to that conversation knows what the missing log line costs.
LLM calls fail. Tool calls fail. Networks blip. Models occasionally return malformed JSON. A production agent has to handle all four without surfacing the failure to the end user.
The minimum pattern: exponential backoff on transient failures, idempotency keys on tool calls that mutate state, a circuit breaker on the LLM call so that a model outage does not melt your downstream stack, and a fallback to a smaller or cheaper model when the primary times out.
import time
from collections import deque
from tenacity import (
retry, stop_after_attempt, wait_exponential, retry_if_exception_type,
)
from anthropic import APITimeoutError, APIStatusError
class LLMCircuitOpen(Exception):
pass
class CircuitBreaker:
"""Opens when failure rate exceeds threshold over a rolling window."""
def __init__(self, window_seconds: int = 60, failure_threshold: float = 0.5,
min_calls: int = 10, cool_down_seconds: int = 30):
self.window = window_seconds
self.threshold = failure_threshold
self.min_calls = min_calls
self.cool_down = cool_down_seconds
self.events: deque = deque() # (ts, ok: bool)
self.opened_at: float | None = None
def record(self, ok: bool) -> None:
now = time.monotonic()
self.events.append((now, ok))
cutoff = now - self.window
while self.events and self.events[0][0] < cutoff:
self.events.popleft()
def is_open(self) -> bool:
now = time.monotonic()
if self.opened_at and now - self.opened_at < self.cool_down:
return True
if self.opened_at and now - self.opened_at >= self.cool_down:
self.opened_at = None # half-open: allow one probe
if len(self.events) < self.min_calls:
return False
failures = sum(1 for _, ok in self.events if not ok)
if failures / len(self.events) >= self.threshold:
self.opened_at = now
return True
return False
circuit_breaker = CircuitBreaker()
@retry(
stop=stop_after_attempt(3),
wait=wait_exponential(multiplier=1, min=1, max=10),
retry=retry_if_exception_type((APITimeoutError, APIStatusError)),
reraise=True,
)
def call_primary(prompt: str) -> str:
if circuit_breaker.is_open():
raise LLMCircuitOpen()
try:
msg = primary_client.messages.create(
model="claude-opus-4-7",
max_tokens=2048,
messages=[{"role": "user", "content": prompt}],
)
circuit_breaker.record(ok=True)
return msg.content[0].text
except (APITimeoutError, APIStatusError):
circuit_breaker.record(ok=False)
raise
def call_with_fallback(prompt: str, idempotency_key: str) -> str:
if cache.has(idempotency_key):
return cache.get(idempotency_key)
try:
result = call_primary(prompt)
except (LLMCircuitOpen, APITimeoutError, APIStatusError):
log.warning("primary_failed_falling_back", key=idempotency_key)
result = fallback_client.messages.create(
model="claude-haiku-4-5",
max_tokens=2048,
messages=[{"role": "user", "content": prompt}],
).content[0].text
cache.set(idempotency_key, result, ttl=300)
return result
What this gets you: 3 retries with exponential backoff on the primary, a circuit breaker that opens when failure rate crosses 50% over a 60-second window, automatic fallback to Haiku when Opus is slow or the breaker is open, and idempotency on the result. None of that is novel engineering. All of it is missing from most agent prototypes.
Anthropic’s guidance is explicit on the cost of skipping this: “The autonomous nature of agents means higher costs, and the potential for compounding errors.” (Anthropic, Building Effective Agents, 2024) Compounding errors is what happens when retry logic is missing. One transient failure becomes a cascade.
What does it cost to skip? Ask the on-call engineer who just got paged at 2am for a transient model timeout that should have retried, or the customer whose action failed because a network blip surfaced as a 500 instead of a quiet retry behind the scenes.
An audit trail is an immutable record of every agent decision and the data it touched. For regulated environments, it is non-negotiable. HIPAA’s Security Rule at 45 CFR 164.312(b) requires covered entities to “implement hardware, software, and/or procedural mechanisms that record and examine activity in information systems that contain or use electronic protected health information.” (Cornell Law, 45 CFR 164.312) That is the regulatory text. An LLM call that touches a patient record is “activity in an information system that contains ePHI.” It needs an audit log.
For non-regulated environments the requirement is the same shape and a different reason. When your agent makes a call that costs a customer money (a refund, a reschedule, a billing adjustment) you want a tamper-evident log of what it decided, what data it saw, and which model produced the output. That log is what lets you reverse a bad decision, settle a dispute, and improve the system.
The architecture pattern we ship: a separate immutable log path, write-only from the agent process, hashed and chained so any tampering is detectable, with retention policies set per-vertical (HIPAA: 6 years minimum from creation or last effective date; SOC 2: typically 1 year operational + 3 years archive depending on auditor; FieldOps SaaS: 90 days hot + 1 year cold for support cases). The log captures: timestamp, agent version, model version, request ID, tenant ID, user ID (where applicable), inputs (with PII redacted or tokenized), tool calls, tool outputs, final decision, and a hash of the prior log entry.
We built this pattern for Paramount Acceptance’s regulated environment (consumer finance, audit-trail rigor adjacent to HIPAA) and adapted it for HealthTech rebuilds. We also built it for Eptura’s multi-tenant workplace platform, where the audit log is what lets a single customer subpoena their own activity history without seeing anyone else’s.
A production audit record is append-only and answers who, what, when, on whose behalf, and why. For a single agent action it looks like this: { "ts": "2026-01-14T09:42:11Z", "actor": "agent:intake-v3", "on_behalf_of": "user:4821", "tenant": "clinic-77", "action": "chart.write", "model": "claude-opus-4.7", "inputs_hash": "sha256:...", "tool_calls": ["ehr.create_note"], "decision": "auto-approved", "reason_code": "confidence>=0.9" }. Write it to storage you cannot edit after the fact (an append-only table or object-lock bucket), so the record survives a dispute. The test: pick any past agent decision and reconstruct exactly what the agent saw and did. If you cannot, you do not have an audit trail. You have a log.
Cost to skip: in a regulated environment, you cannot ship. In an unregulated environment, you ship and the first dispute kills the agent’s credibility internally.
Compliance baked into the agent architecture, not bolted on post-hoc. This is where the layers compound: eval harness verifies the agent meets a clinical-safety bar, observability gives you the per-call evidence, error handling stops silent failures, audit trails satisfy the regulator. Compliance integration is the architectural pattern that ties the four together.
Concretely, compliance integration means mapping each agent capability to the controls it touches, then enforcing them in code rather than in a policy doc. For a HIPAA agent that is four rows: PHI in prompts → field-level redaction before the model call; PHI in logs → tokenize before write (Layer 2 feeds this); data residency → pin inference to a BAA-covered region; access → scope every tool call to the authenticated user’s tenant (Layer 4 records it). Each row has an owner and an automated test, not a checkbox.
What it looks like by vertical:
HealthTech (HIPAA, HITRUST). BAA in place with every model provider before a production call goes out. PHI redaction or tokenization on tool inputs and outputs. RBAC on tool access (the intake agent cannot call the prescribing tool). De-identification on eval datasets so you are not training or evaluating on real PHI. Human-in-the-loop gates before an agent writes to the EHR. We walk through the full HealthTech pattern in our companion piece, Production-Grade AI Agents in Healthcare: HIPAA Architecture. Read it after this if your vertical is HealthTech.
FinTech / InsurTech (SOC 2, PCI-DSS, model-risk review). Explainability captured per decision so a regulator or internal model-risk team can reconstruct the chain of reasoning. Tokenization on payment data before it reaches the model. Fair-lending tests in the eval harness when the agent touches credit decisions. Documentation that maps to NIST AI RMF. The framework defines four core functions for trustworthy AI: Govern, Measure, Manage, and Map. (NIST AI Risk Management Framework) Most model-risk reviewers we have talked to are using the RMF Playbook as their evidence rubric.
FSM Vendor / multi-tenant SaaS. Tenant isolation in the agent’s tool layer. No agent should be able to read across tenants. Per-tenant rate limits so a single noisy tenant cannot starve the rest. Per-tenant audit logs (Layer 4 with strict tenant-scoped read access). Eval harness includes adversarial cases for cross-tenant leakage.
FieldOps Operator / low-tech end users. The compliance-equivalent here is not regulatory. It is reliability for users who cannot debug. If the dispatch agent hallucinates a job assignment, a $10M plumbing operator loses a day’s revenue. The eval harness focuses on dispatch-correctness, and the audit trail is what lets a dispatcher say “the agent moved my crew at 9:14, why?” For the FieldOps Operator-side view of these patterns (translated for an owner-CEO who doesn’t speak architecture), see what custom dispatch software actually looks like for an operator buyer.
A 2025 survey of LLM agent evaluation organizes the work along two axes: what to evaluate (behavior, capabilities, reliability, and safety) and how to evaluate (interaction modes, datasets, metric computation, tooling), and explicitly calls out enterprise-level concerns like role-based data access, reliability guarantees, dynamic long-horizon interactions, and compliance. (Yehudai et al., Evaluation and Benchmarking of LLM Agents: A Survey, arXiv:2507.21504, 2025) Compliance integration, in practice, means picking the dimensions that matter for your regulated environment and testing them every commit. Skip this layer and you ship a feature, your security team or your auditor blocks the launch, and you spend 3 months retrofitting compliance into an architecture that was not designed for it. Retrofit is always more expensive than build-in, and the production AI agent that made it past staging without compliance baked in is the one that gets pulled before launch.
The order to ship a production AI agent from a working prototype is: eval harness first (always), then error handling and retries (Layer 3), then observability (Layer 2), then audit trails (Layer 4, go directly here if you’re in a regulated vertical). Compliance integration (Layer 5) runs in parallel with all four. Each layer takes 1-2 engineer-weeks for the first version. A team that ships all five in 8-12 weeks is moving fast. A team that ships in four weeks shipped a demo and will find out about the missing layers in production.
Eval harness first. Always. Without an eval harness you cannot verify that any of the other layers actually move the reliability needle. 5–10 test cases per agent capability, schema validation, recall and precision metrics on the structured output. Two engineer-weeks for the first version.
Then Layer 3 (error handling). Retries, idempotency, fallbacks. One engineer-week if your tooling already supports it.
What about observability? Layer 2 comes next. Structured logs on every step, per-tenant attribution, dashboards for the on-call team. One to two engineer-weeks.
Then Layer 4 (audit trails), but skip directly to it if you are in a regulated vertical. HIPAA, PCI-DSS, and SOC 2 environments cannot launch without it. One to two engineer-weeks for the immutable log path.
Layer 5 (compliance integration) is continuous. It informs every decision in Layers 1–4. If your vertical is regulated, you start Layer 5 thinking on day one, even if the implementation lags.
The team that ships all 5 layers in 8–12 weeks is moving fast. The team that ships them in 16–24 weeks is moving normally. The team that ships in four weeks shipped a demo and is going to find out about the missing layers in production.
This is the work behind the tagline. From Prompt to Production.
How long does each layer take to build?
For a typical agent in a 50–500 employee company: Layer 1 (evals) 1–2 weeks, Layer 2 (observability) 1–2 weeks, Layer 3 (error handling) 1 week, Layer 4 (audit trails) 1–2 weeks, Layer 5 (compliance integration) ongoing. Total: 8–12 engineer-weeks for a production AI agent if the team has done it before. 16–24 weeks if it is the team’s first production AI agent.
Can I add evals after launch?
Technically yes, practically no. Once an agent is in production, every prompt change is a regression risk. Adding evals retroactively means cataloging the behaviors you already shipped, which is harder than capturing them as you build. Build the eval harness first, even if it is the smallest possible version (5 examples per capability).
Do I need all 5 layers for an internal tool?
No. An internal tool used by 10 engineers needs Layer 1 (evals) and Layer 3 (error handling). Layer 2 (observability) becomes important when a different team owns it. Layer 4 (audit trails) is required when the tool touches regulated data or money. Layer 5 (compliance integration) is required when the tool faces regulators or external auditors. Match the layer set to the risk surface.
What is the difference between an agent prototype and a production agent?
The prototype works on the happy path with one user and clean inputs. The production agent works on the happy path, the unhappy paths, the adversarial inputs, the network blips, the model outages, the noisy tenants, the regulated audits, and the 2 AM incidents, for thousands of users across multiple tenants. Same model. Different system around it.
Are these 5 layers the same for every vertical?
The layers are the same. The implementation specifics shift. HealthTech needs the layers plus PHI handling and BAAs. FinTech needs the layers plus model-risk documentation. FSM Vendor needs the layers plus multi-tenant isolation. FieldOps Operator needs the layers plus reliability framing for low-tech end users. Build the 5 layers in a vertical-aware way; do not pretend a generic agent stack will satisfy a regulated environment.
Deeper looks at the buy-or-build, integration, and outgrowth decisions in this space:
If you have an agent prototype and a production launch on the calendar, book a 15-min discovery call: https://cal.com/mirgen-motomtech/quick-intro. Tell us which layer you are stuck on. We will walk you through the architecture pattern we have shipped for similar agents and tell you honestly whether the work is a fit for our team or yours.
