A good AI agent says “not me” a lot.
Most AI demos reward the agent for answering everything. The agents worth trusting know their lane, answer inside it and pass the rest to a person. How to draw the lane, and how to check it’s holding.
By Mathieu Sroussi, founder of Syrup
This year we wrote the performance terms for an email agent. One shared mailbox. Most of the traffic was the same question, asked forty different ways: “where is my certificate?”
The target was for the agent to handle 80% of the emails in its lane. But the clause that mattered most was a different one. When the agent hands an email to a person because the rules say so (a complaint, a legal threat, a language it doesn’t support), that counts as a success. Not a miss. A success.
That one line changes how you build an agent.
The demo problem
Most AI demos are scored on coverage: how many questions did it answer? That rewards the agent for always having a go. In a demo, a confident wrong answer looks the same as a right one. In front of a customer, it’s a tribunal case. (See: the airline that argued its chatbot was responsible for its own words.)
An agent rewarded for handing over learns the opposite habit. It answers what it’s sure of, and passes the rest on, with a note.
Draw the lane first
Before anyone writes a prompt, write two lists.
What the agent may handle. Be specific. “Certificate requests for active policies, where the client is found and the document exists.” Not “customer queries”.
What it must never touch. Ours usually start like this:
- Complaints, or anything that sounds like one.
- Legal threats, a mention of the ombud, “my lawyer”.
- Bank details, payments and refunds.
- Cancellations, and changes to cover or terms.
- Any language the agent hasn’t been tested in.
- Any case where it can’t find the record, or finds two.
- Any judgement about a person: affordability, eligibility, suitability. Section 71 of POPIA limits decisions about people based solely on automated processing.1 Keep a person in those.
The second list matters more than the first. Every item on the second list is a sentence the agent will never write.
Make the hand-over useful
“Escalated to a human” is only half the job. A good hand-over arrives with:
- what the email is about, in one line;
- why the agent stopped, and which rule it hit;
- what it found: the client, the policy, the last three emails;
- a draft, if one is safe to suggest, clearly marked as a draft.
The person picks it up in thirty seconds instead of five minutes. A good share of the time saved comes from here: not only the emails the agent answers, but the ones it prepares.
Check it every week, boringly
Our quality check is dull on purpose. Every week, one named person reads twenty of the agent’s replies and hand-overs. Four questions, pass or fail:
- Did it read the email right?
- Did it pick the right reply, or rightly hand over?
- Right language, right tone?
- Nothing made up?
Fifteen minutes, and a running score. When something fails, the rule or the source document gets fixed that week.
(Nobody gets excited about a rubric. That’s usually the sign of a good one.)
Earn the right to send
Start with the agent drafting and a person sending everything. After a few weeks of passing checks, let it send the simplest category on its own: the certificate for an active policy, say. Keep everything else in draft. Widen the lane only when the weekly check says you can.
That’s slower than a demo. It’s also how an agent is still switched on in month six.
So, if an agent started answering your customers tomorrow, which emails should it never touch? Write that list first.