How to create an AI agent that survives production

1. Decide whether the case requires an agent or pipeline

Agent is expensive in latency, cost and failure surface. Before writing the first line, check whether the problem has real decision ramifications. If the flow is always A then B then C, you want a deterministic workflow with one or two LLM calls inside it — not an agent.

The practical criterion: if you can draw the complete flowchart without using the word "it depends", you are not an agent. An agent is justified when the number of possible paths is too large to code and the cost of making mistakes is recoverable.

• Pipeline: data extraction, classification, summarization, batch enrichment.
• Agent: triage with follow-up, support that queries multiple systems, iterative search, operations that require verification before writing.

2. Model the tools before the prompt

Agent capabilities are exactly the set of functions that you expose. Each function needs typed schema, description written for the model (not pro dev), input validation, and readable errors — the model reads the error message and tries again, so a "500 Internal Server Error" wastes an entire cycle.

Separate reading from writing. Reading tools can be released; writing tools go through confirmation, idempotency key and audit log.

const buscarPedido = {
  name: "buscar_pedido",
  description:
    "Retorna status, itens e prazo de um pedido. Use quando o cliente citar número de pedido.",
  parameters: z.object({
    pedidoId: z.string().regex(/^\d{6,10}$/),
  }),
  handler: async ({ pedidoId }, ctx) => {
    const pedido = await db.pedido.find(pedidoId, { tenant: ctx.tenantId });
    if (!pedido) {
      return { erro: "Pedido não encontrado para este cliente. Peça o número novamente." };
    }
    return pick(pedido, ["status", "itens", "prazoEntrega"]);
  },
};

3. Choose the control loop

Three topologies cover almost everything. Single agent with tools solves 80% of cases and is what you should try first. Orchestrator with specialized sub-agents is useful when domains have prompts and tools that are incompatible with each other. Explicit state graph is valid when the process has mandatory auditable steps.

In any topology, impose an iteration ceiling, timeout per tool and global timeout. A homeless loop agent is an incident awaiting date.

4. Memory: context, session and knowledge

Confusing the three types of memory is the most common cause of expensive and confusing memory. Context is the current call window and should be mounted, not accumulated. Session is the state of the conversation, saved in the bank and summarized when it passes a limit. Knowledge is what the agent seeks on demand via retrieval.

For knowledge, RAG with embeddings solves semantic search in documentation; direct SQL query solves structured data and is cheaper, faster and auditable. Many people put what should be a SELECT in a vector.

• Context: system prompt + last N messages + round tool results.
• Session: persisted history, rolling summary, entities already identified.
• Knowledge: pgvector for text, relational database for facts, cache for what repeats.

5. Guardrails and security

Treat all model output as untrusted input. The agent can be tricked by content they read — an email, a page, a customer PDF — into calling tools you didn't intend. The defense is not a better prompt, it is authorization in the handler.

Each call loads the tenant and user from the execution context, never arguments from the model. Data scope is resolved on the server. Destructive operations require human confirmation or go through an approval queue.

• Never accept tenantId, userId or role as a tool parameter.
• Rate limit per session and per account, not just global.
• PII filter in log entry and what is sent to the provider.
• Kill switch per tool, activatable without deployment.

6. Evaluation: what separates engineering from trial and error

Without an evaluation set, you cannot change models, adjust prompts or reduce costs without risk. Assemble 30 to 100 real cases with expected results, run in CI with each change and monitor three metrics: task success, correct choice of tool and average cost per execution.

For open-ended output, use a judge template with written rubric and weekly human sampling. Don't chase 100%: set the acceptable threshold and fallback behavior below it.

7. Cost and latency as a design requirement

Agent cost grows non-linearly: each iteration reloads the entire context. Two levers solve most of this — reduce what goes into context and route by complexity, with a small model in triage and a large model only when the task requires it.

Measure cost per conversation resolved, not per token. It is the only metric that compares to the cost of the manual process that the agent replaces.

• Prompt cache for stable system prompt.
• Streaming for latency awareness, with tools in parallel when independent.
• Spending cap per session, with controlled degradation instead of a surprise bill.

8. Operation: observability and rollout

Each execution needs a complete trace: messages, tools called, arguments, latency, tokens and outcome. Without this, debugging agents becomes archeology.

Climb with small scope and clear escape path. An agent who passes the baton to a human when he is unsure generates more trust than one who answers anything with conviction.