The frameworks are not the hard part. You can stand up a working agent on LangGraph or CrewAI in an afternoon, and it will do something impressive in a meeting.
The hard part is everything around the loop: what happens when a tool times out, what stops a confused agent spending four hundred dollars in tokens overnight, who approves the action that emails a customer, and what you read at two in the morning when something went wrong.
The gap between a demo and a deployment
A demo agent has a cooperative environment. The API responds, the record is clean, the task matches what the developer had in mind. A production agent meets rate limits, partial outages, records with contradictory data, and inputs nobody anticipated.
Almost every agent we have been asked to take over failed on the same handful of things, and none of them were about model quality.
An agent without a step limit is a loop with a credit card.
Bounds: stop the loop before it stops you
The first three checks are about making the run finite and affordable.
- A hard step limit. Every graph needs a maximum number of transitions and an explicit terminal state for hitting it. Without one, an agent that cannot satisfy its goal will circle indefinitely.
- A cost ceiling per run and per tenant. Track tokens as you go and abort with a clear error when the run exceeds its budget. Report cost per run from the first sprint so it is a design input rather than a month-end surprise.
- A wall-clock timeout. Separate from the step limit, because a single slow tool call can hang a run that has taken only three steps.
Tool contracts: never let a failure look like data
This is the failure that produces the most confident wrong answers. A tool call times out, the wrapper returns an empty string or a null, and the agent reasons from it as though it were a real result. The output is fluent, internally consistent and completely fabricated.
Every tool the agent can reach needs a typed interface with three things: validated input, validated output, and an explicit error state that is distinguishable from a successful empty result. "No records found" and "the query failed" must not look the same to the agent.
Give tools least-privilege credentials scoped to what that specific agent needs. An agent that only reads should hold a read-only key, so a prompt injection in a document cannot turn into a write.
Approval gates: consequence decides, not confidence
The question is not whether the agent is usually right. It is what happens on the occasions it is wrong.
Route through a human queue anything that spends money, contacts a customer, changes a system of record, grants access or cannot be reversed. Everything else can run unattended from day one. The queue should show the proposed action alongside the reasoning and the evidence, so review takes seconds rather than requiring reconstruction.
Gates come off one at a time, and only when the data supports it. If the agent's proposed action has been approved unchanged 200 times in a row for a given action type, that is an argument for removing the gate on that type. A general feeling that it is working well is not.
Observability: the trace is the product
When an agent misbehaves, the question is always "what did it do and why". If the answer requires reconstruction from application logs, you will not answer it.
Log every run end to end: inputs, each tool call with its arguments and response, intermediate reasoning, the final output, tokens, latency and cost. Make it searchable, because the useful query is "show me every run where the CRM update failed last week", not "show me this one run I already know about".
Then alert on the aggregate rather than the individual: failure rate by tool, approval-rejection rate by action type, cost per run trending up, step count distribution shifting. Those are the signals that tell you something changed upstream.
Rollout: shadow first, then a slice
Run the agent alongside the existing process without acting for long enough to accumulate real comparisons. You are looking for where its decisions diverge from the human ones and whether the divergence is the agent being wrong or the human being inconsistent. Both are informative.
When the gap is small enough, roll out by team or by queue rather than all at once, and keep the previous process available for the first few weeks. Reverting should be a configuration change, not an incident.
On one autonomous support deployment this staged approach ended with 61% of queries resolved without escalation and average response time down 44%, with sentiment-based escalation carrying everything that needed human judgement. measured · support-platform · prod
The checklist
Before an agent runs unattended against production systems:
- Hard step limit with an explicit terminal state
- Cost ceiling per run and per tenant, enforced not just monitored
- Wall-clock timeout independent of the step limit
- Typed input and output contracts on every tool
- Explicit error states distinguishable from empty results
- Least-privilege credentials scoped per agent
- Untrusted content treated as data, never as instruction
- Approval gates on anything irreversible or customer-facing
- Full run traces, searchable, with tokens and cost
- Aggregate alerting on failure rate, cost trend and rejection rate
- A shadow period with recorded divergence from the current process
- A documented rollback that is a configuration change
None of this is exotic. It is the same discipline any other unattended production system gets, applied to a component whose failure modes are less familiar and more expensive to discover in the wild.
Key takeaways
- Step limits, cost ceilings and timeouts make a run finite and affordable before anything else matters.
- A tool failure must never be indistinguishable from a valid empty result.
- Gate on consequence, not on confidence, and remove gates using approval data rather than instinct.
- Searchable end-to-end traces are what make an agent operable rather than merely functional.
- Shadow running against the existing process is the cheapest way to find out whether it is ready.