Skip to content
ZJ

Case study /05 · Private beta

Autonomous Support Agent

AI · Agents · Customer Automation

A tool-using support agent that resolves real tickets end to end — and knows exactly when to hand off to a human.

The Autonomous Support Agent handles customer conversations the way a trained support rep does: understand the issue, pull up the account, apply policy, act — refund, reship, update — and write it all down. The difference is that its permissions are enforced by an engine, not by trust.

  • TypeScript
  • Node.js
  • LLM tool use
  • Function calling
  • RAG
  • PostgreSQL
  • Redis
  • Guardrails
  • Evals
Autonomous Support Agent end-to-end flow: a customer message is understood and planned, context gathered, an action decided against policy, permitted tools executed and the result verified — with a human approval checkpoint for consequential actions and escalation when the agent is uncertain.

Context

LLM support bots that only answer FAQs move no needles; the cost lives in tickets that require account lookups and actions. Tool-using agents can close that class of ticket — if the action layer is engineered with the same care as a payments system.

Problem

An agent with real tools is an agent with real blast radius. The system had to reason over messy customer language, choose correct tool sequences, respect hard policy limits (refund caps, identity checks), recognize its own uncertainty, and escalate with full context instead of failing silently.

Constraints

  • Consequential actions (refunds, account changes) carried hard, engine-enforced limits and identity verification requirements.
  • Every action needed an audit trail linking conversation, reasoning, tool call and result.
  • Escalation to humans had to carry full context — no customer repeats their story.
  • The agent had to be evaluated against historical tickets before ever touching a live customer.

Responsibilities

  • Agent loop design: planning, tool selection, result interpretation and recovery.
  • Typed tool layer over order, account, refund and knowledge systems.
  • Policy engine defining allowed, approval-required and forbidden action classes.
  • Escalation and handoff flows into the human support queue.
  • Simulation harness replaying historical tickets for pre-deployment evaluation.

Architecture

  • The agent runs a bounded reasoning loop: classify intent, gather context via read-only tools, propose actions, execute permitted ones, verify results.
  • Tools are typed contracts with declared action classes; the execution engine enforces policy mechanically — a forbidden call cannot execute regardless of model output.
  • Read paths and write paths are asymmetric: reads are free, writes pass through policy checks, limits and (where required) customer identity verification.
  • Conversation state and tool results live in a structured session store, which is also what a human receives on escalation.
  • Every step appends to an immutable audit log tying language to action.

Key decisions

  • Policy in the engine, not the prompt — the same principle as Phandor, applied to customer operations.
  • Uncertainty triggers escalation, not guessing: low-confidence intent or contradictory account state routes to humans early.
  • Simulation before production: the agent replayed months of historical tickets, and its proposed resolutions were graded against what actually resolved them.
  • Small tool surface first — a handful of reliable tools resolved the bulk of automatable tickets; breadth came later.

Implementation

  • Refund and account tools enforce per-customer and per-period caps below the model layer.
  • Escalations package the conversation, gathered context and attempted actions into the support desk as a structured case.
  • Regression evals run on tool or prompt changes, scoring resolution correctness, policy compliance and escalation precision.
  • Response streaming keeps perceived latency conversational while tools execute.

Performance

  • Read-tool parallelism keeps account-context gathering inside a conversational pause.
  • Session state in Redis keeps multi-turn context retrieval flat under load.
  • Bounded loops cap worst-case reasoning cost per ticket.

Outcomes

  • An agent that resolves routine account tickets end to end within mechanically enforced policy.
  • Escalations arrive with complete context, making the human queue faster too.
  • The simulation harness caught policy-violating behaviors before any customer ever saw them.
  • [PLACEHOLDER — add resolution rates and volume figures when publishable.]

Lessons

  • Agent quality is bottlenecked by tool design more than model choice.
  • Escalation is a success path, not a failure path — design it first.
  • Replaying history is the cheapest honest evaluation an agent can get.

Software Architecture

Six-stage flow from customer message to resolution or human handoff. 1. Customer message, arriving by web, email, chat or ticket. 2. Understand and plan: classify intent and choose next steps. 3. Gather context: read account, order and policy data. 4. Decide action: propose a resolution and check policy. 5. Execute tools: run a permitted refund, reship or update. 6. Verify and resolve: confirm the result, respond and close. A human approval checkpoint branches off the decide and execute stages and is required for consequential actions; when the agent is uncertain or an action fails, the path routes to human escalation carrying full context, attempted actions and the reason, then rejoins the verify stage. An immutable audit trail records conversation, reasoning, tool call and result. Four properties are called out: engine-enforced permissions, human handoff, auditability and policy-safe actions.
Architecture diagram of typed tools, engine-enforced policy, state, retrieval and evaluation, in five columns. Channels: web and chat, email and ticket, voice and SMS, and a human desk. Agent orchestration: session manager holding state and memory, intent classifier to understand the issue, planner to choose tools, and a recovery loop that retries or escalates. Tools and RAG: read tools for profile, order and policy, write tools for refund, reship and update, knowledge RAG over policies and FAQs, and tool contracts with typed inputs and outputs. Policy and safety: action classifier splitting actions into allowed, approval-required and forbidden, policy enforcer applying caps, limits and rules, identity check required before sensitive writes, and escalation on low confidence or conflict. Data: Postgres for tickets and audit, Redis for session and queues, evals holding replay history, and alerts on policy failures. An observability and evaluation band spans all five: immutable audit log, historical ticket simulation, regression evals, metrics and alerts.