Building Production AI Agents

Most agent demos are optimized for one thing: showing that the model can complete a task once. Production systems have a different requirement. They need to complete tasks repeatedly, under changing inputs, with bounded risk, observable behavior, and recoverable failure modes.

That difference changes the architecture.

An agent in production is not a prompt with tools attached. It is a software system with a probabilistic component inside it. The useful engineering work happens around that component: constraining it, observing it, validating it, and deciding which parts of the workflow should not be left to the model.

Why production agents are different from demos

A demo can tolerate ambiguity. A production system cannot.

In a demo, it is acceptable if the model invents a step, retries manually, or produces a plausible answer. In production, every action has a cost:

• a tool call may mutate external state;
• a wrong classification may route work to the wrong system;
• an unbounded retry loop may create duplicate operations;
• a missing trace may make an incident impossible to understand;
• a confident answer may hide uncertainty from the user.

The core design question is therefore not:

Can the model do this?

It is:

Can the system make this task safe, inspectable, and repeatable?

That framing is more useful because it forces the architecture to include constraints from the beginning.

Architecture

The most reliable agent systems I have worked with had explicit boundaries between reasoning, execution, state, and verification. When those boundaries were blurred, debugging became transcript archaeology.

LLM layer

The model should not be treated as a general-purpose control plane.

It is better used as a reasoning and transformation component: classify intent, extract structured data, propose plans, summarize context, or decide between bounded options. The more open-ended the model's authority, the harder it becomes to reason about the system.

Useful model calls tend to have:

• narrow prompts;
• explicit schemas;
• small decision surfaces;
• clear fallback behavior;
• examples of invalid behavior, not only valid behavior.

The schema matters more than the prose. A well-designed output contract gives the rest of the system something deterministic to validate.

Tool execution

Tools are not just functions. They are authority boundaries.

A tool should describe what it can do, what it cannot do, what state it may change, and how the system can safely retry or compensate if something fails. The agent should not call tools directly as an unstructured side effect of text generation.

Good tool design usually includes:

• typed inputs and outputs;
• idempotency keys for mutating operations;
• permission checks outside the model;
• dry-run or preview modes for risky actions;
• audit logs for every execution;
• clear error categories.

The model can request an operation. The system should decide whether that operation is allowed.

State management

Agents need memory, but memory is a dangerous word.

There are at least three different kinds of state:

1. conversation state;
2. task execution state;
3. long-term user or business context.

Mixing them creates bugs. A chat transcript is not a durable workflow state machine. It may contain useful context, but it should not be the only source of truth for what has happened.

For production workflows, the system should record:

• what task was requested;
• which plan was selected;
• which tools were called;
• what external state changed;
• what remains unresolved;
• whether the result was verified.

This makes retries and support possible. Without it, the system depends on re-reading the model's previous messages and hoping the transcript is enough.

Memory

Long-term memory should be boring.

The temptation is to store everything and let retrieval solve the problem later. That usually creates noise. A better approach is to store stable facts, decisions, preferences, and previous outcomes with source information and freshness.

Memory entries should answer:

• where did this fact come from?
• when was it last verified?
• who or what is allowed to use it?
• what should happen if it conflicts with newer information?

Without those answers, memory becomes a second prompt: useful in demos, unreliable in production.

Error handling

Agent systems need error handling at every layer.

Model errors are only one class of failure. The more common failures are ordinary distributed system failures: timeouts, partial writes, stale data, invalid responses, permission mismatches, duplicated requests, and external APIs changing behavior.

The system should distinguish:

• model uncertainty;
• schema validation failure;
• tool execution failure;
• permission failure;
• external dependency failure;
• verification failure.

Those categories should produce different behavior. Retrying a permission failure is noise. Retrying a transient dependency failure may be correct. Asking for human confirmation may be better than letting the model guess.

Observability

Logs alone are not enough.

An agent trace should show the whole execution path: input, classification, selected plan, model calls, tool requests, tool results, validation failures, retries, and final response. This is not only for debugging. It is also for trust.

When a user asks why the system did something, the answer should not be "because the model said so." The answer should be visible in the trace.

Lessons learned

Agents need constraints

Unconstrained agents look more capable than constrained agents until they fail.

Constraints are not a limitation of the product. They are the reason the product can be operated. Budgets, permissions, schemas, tool contracts, and verification steps make the system less magical and more useful.

Tool design matters more than prompts

Prompt quality matters, but tool design has more leverage.

A vague tool with broad authority will create unreliable behavior even with a good prompt. A precise tool with a narrow contract gives the model a safer surface to operate on.

The best tools are boring. They expose small operations, return structured results, and make failure explicit.

Reliability requires engineering around the model

The model is not the system.

Reliability comes from the surrounding architecture: deterministic workflow steps, durable state, validation, idempotency, observability, and human review where appropriate. The model can be powerful inside that structure. Outside of it, it becomes difficult to trust.

The useful mental model is simple:

Let the model reason. Let the system enforce.

That division is not always clean, but it is a good starting point.