How to build a customer-support AI agent for SaaS

Why support agents are the SMB's wedge into AI

Customer-support volume scales linearly with revenue, but the answers don't. 70% of tier-1 tickets are repeat questions your docs already answer — but no user reads docs. An AI agent that reads the docs for them deflects that 70%, and the back half (the ones humans should actually touch) gets routed cleaner.

The docs problem nobody budgets for

A retrieval-grounded agent is a function of its corpus, and most SaaS documentation is not in a state to be that corpus. This is the single most common reason a support-agent project lands late.

Docs written for browsing, not for answering. A page titled "Getting Started" that covers eight unrelated tasks is a poor retrieval unit. Whichever chunk the search returns will be mostly irrelevant to the question, and the model will either answer from the wrong part or refuse. Documentation that reads well as a guided tour frequently retrieves badly.

Contradictions nobody noticed. Two pages describing the same limit with different numbers is invisible while humans read one page at a time, and immediately visible once a retriever surfaces both. Expect to find a handful of these, and expect the answer to require a product decision rather than an edit.

Undocumented truth. A significant share of tier-1 answers live in support macros, old tickets, and one engineer's head rather than in the docs. Mining resolved tickets for the answers that were given repeatedly is usually the fastest corpus improvement available, and it is work that pays off whether or not the agent ships.

Versioning. If your product has plans or releases that behave differently, the corpus has to carry that distinction, and the retriever has to know which one the asking user is on. An agent confidently describing a feature the user's plan does not include produces a worse support outcome than silence.

Budget the cleanup explicitly. Teams that treat it as a prerequisite ship on schedule; teams that treat it as a surprise discover it in week three and lose the time anyway.

The architecture we ship

A production support agent has six moving parts. Most "AI chatbot" attempts skip three of them and wonder why the thing hallucinates.

1. Intent classifier — routes the ticket into resolvable / needs-human / docs-gap / feature-request.
2. Retrieval — hybrid search (BM25 + vector) over your docs, with reranking. Citations required.
3. Answer generation — Claude or GPT, prompted to refuse when retrieval confidence is low.
4. Escalation path — a confidence threshold and a human queue. Non-negotiable.
5. Eval loop — 200 historical tickets replayed weekly. Drift kills support bots quietly.
6. Cost telemetry — every conversation logged with token cost, deflection outcome, CSAT.

Skip the eval loop and you'll ship a great demo that's silently degraded in 90 days.

Designing the escalation, not just the answer

Most of the design effort goes into what the agent says. Most of the customer experience is decided by what happens when it stops.

The handoff has to carry context. An escalation that dumps a customer into a fresh chat with a human, having lost the conversation, is worse than no agent — the customer has now explained themselves twice. The transfer needs to carry the transcript, the retrieved sources the agent considered, the account context it looked up, and the reason it gave up. That last field is the one teams forget, and it is the most useful thing an agent hands a human.

Escalate on the right signal. Retrieval confidence is the obvious trigger and an incomplete one. The others worth wiring: explicit user frustration, a second failed attempt at the same intent, anything touching billing or cancellation, and any question where the answer would commit the company to something. That last category should escalate regardless of how confident the model is.

Make the exit visible from the first message. A visible route to a human reduces frustration even when it goes unused, because the cost of the agent being wrong is now bounded. Hiding it to protect deflection numbers reliably backfires; users who feel trapped escalate on public channels instead.

Staff the queue before you launch. An escalation path that lands in an unmonitored inbox converts every agent failure into a lost customer. If the human queue is not ready, the agent is not ready either.

Scope it by intent, not by percentage

The most useful decision on these projects is made before any code exists, and it is not about the model.

Teams typically scope a support agent as a percentage — "deflect 40% of tickets" — which is a target, not a specification. It gives the build no boundary, because every ticket is theoretically in scope and the agent is being asked to be good at everything at once.

Scope by intent instead. Pull a month of tickets, cluster them, and rank by volume. In most B2B SaaS the top five intents account for well over half of tier-1 volume, and they are usually mundane: password and access, billing status, a handful of how-to questions, integration setup, and "is this thing down." Build for those five, explicitly and well. Everything else routes to a human by design rather than by failure.

This produces a smaller, better system and an honest launch conversation. "It handles these five things reliably and hands over everything else" is a claim you can verify before release and defend afterwards. "It deflects 40%" is a number you discover after launch, usually by disappointing someone.

The percentage then falls out of the intent coverage as a result rather than serving as a goal — which is the right way round, because it is the customer experience on those five intents that determines whether anyone trusts the sixth.

The hard parts

The model is not the hard part. The hard parts are:

• Knowing when not to answer. A good support agent says "I'm not sure — let me route this to a human" more often than it answers.
• Multi-turn state. "I tried that, didn't work" needs to remember what 'that' was.
• Tool use. The agent often needs to look up the user's account, recent activity, or feature flags before it can answer. That means scoped read access to your systems.

What to measure once it is live

Deflection is the number everyone quotes and the easiest one to flatter, so measure it alongside the three that keep it honest.

Containment versus resolution. A conversation that ended without a human is contained. A conversation where the customer got what they needed is resolved. The gap between those two is where abandoned, frustrated users live, and a system optimised on containment alone will happily grow that gap. Track both, and treat a widening spread as a defect.

Reopen rate. A ticket the agent closed that the customer reopens within 48 hours was not deflected, it was deferred — usually with an unhappy customer attached. This is the most reliable single indicator that a deflection number is overstated.

CSAT split by path. Score agent-only conversations separately from escalated ones. If agent-only CSAT is materially below escalated CSAT, the agent is answering things it should be handing over, and the confidence threshold needs raising.

Cost per resolved contact, not per conversation. Token cost per conversation is easy to measure and misleading on its own, because the cheap conversations are frequently the useless ones. Divide by resolutions, not by sessions.

Set the baseline before launch. Deflection means nothing without the pre-agent numbers to compare against, and reconstructing that baseline afterwards is never as clean as measuring it in advance.

What this costs to build

For a B2B SaaS with a few hundred tickets/week and a moderate docs base, a Fixed-fee Sprint runs 5–8 weeks and includes the eval loop and the first three months of tuning. After that, most clients move to a Monthly Retainer for ongoing eval coverage as the product changes.

If your docs are a mess, add 2 weeks for a docs cleanup pass — without it, the agent's only as good as what it can retrieve.

The first ninety days

The rollout sequence matters more than the model choice, and the pattern that works is the same across every engagement we have run.

Weeks 1–2, shadow mode. The agent answers every incoming ticket, and no customer sees a word of it. Support staff work normally and rate the agent's proposed answer alongside their own. This gives you a real accuracy number on real traffic with zero risk, and it surfaces corpus gaps faster than any amount of pre-launch testing.

Weeks 3–4, suggest mode. The agent drafts, a human edits and sends. Handle time drops immediately, which buys goodwill from the support team — and their edits are the highest-quality training signal you will ever collect, because each one is a correction on a real case.

Weeks 5–8, narrow autonomy. Let the agent answer unaided on the two or three intents where shadow mode showed the highest accuracy, usually password, billing-status, and how-to questions. Everything else stays in suggest mode. Resist the pull to widen this quickly; the intents you are tempted to add are the ones with the worst measured accuracy.

From week 9, widen on evidence. Each intent graduates to autonomy on its own numbers, not on the system's average. Some intents never graduate, and that is a successful outcome rather than a failure.

The teams that skip shadow mode to launch faster spend the saved weeks on incident response instead. It is the same time, spent worse, in front of customers.

Where these agents actually go wrong

The failures follow a short list, and none of them is the one people worry about beforehand. Hallucinated facts are the fear; the real problems are quieter.

Confidently answering a stale document. The agent retrieves accurately, cites correctly, and repeats a limit that changed two releases ago. Everything about the response looks right, which is precisely why nobody catches it until a customer acts on it. Corpus freshness is a bigger risk than model accuracy, and it degrades silently.

Answering the question that was asked instead of the one that was meant. "How do I delete my data?" from a user mid-cancellation is a retention conversation wearing a technical question. A literal answer is correct and commercially awful. Some intents need routing on context rather than on wording.

Drifting into commitments. "That should be in the next release" is a sentence no support agent should be able to produce, and models produce it readily because it is the helpful-sounding thing to say. Roadmap, refunds, and exceptions to policy need explicit refusal patterns, not confidence thresholds.

Degrading invisibly after a docs migration. A documentation restructure that improves the site can wreck retrieval overnight — new URLs, changed headings, re-chunked content. Nobody thinks to re-run the eval suite after a docs change, because docs are not code. Wire the re-index and the eval run into that pipeline anyway.

Winning on the wrong metric. An agent tuned to maximise deflection learns that the safest way to avoid escalation is to give a plausible answer and end the conversation. The numbers improve while the experience gets worse. This is the failure that survives longest, because the dashboard is reporting success throughout — which is the whole argument for measuring resolution and reopen rate alongside it.

Start with deflection math

Before you build, do the math. Tickets per week × avg handle time × loaded support cost = the dollar volume of tier-1 you're paying for. A working support agent deflects 30–60% of that. If the deflection dollars beat the build + ongoing ops cost in under 9 months, build it. If not, the docs cleanup might be the better project.