The fastest way to lose a customer’s trust is an AI agent that keeps trying when it should have stopped. Escalation rules decide when the agent hands a conversation to a person, and they deserve as much design attention as the agent’s answers.
Four signals that should trigger a handover
- Low confidence. The agent cannot find a relevant source, or its answer is not grounded in one. Set a threshold and test it against real conversations.
- High stakes. Refunds above a limit, account closures, legal complaints, medical or financial advice. Some topics should always reach a human, however confident the model is.
- Customer frustration. Repeated questions, negative sentiment or an explicit “talk to a person”. Never make customers argue with a bot to reach a human.
- Loops and dead ends. The same tool failing twice, or three turns without progress. Cap the attempts and move on.
Write the rules down, then test them
Escalation logic should not live only in a prompt. Keep the hard rules, such as topics, limits and keywords, in code or configuration where they are visible, versioned and testable. Use the model for the softer judgements, like sentiment, and log every decision it makes.
Build a test set of past conversations that should and should not escalate. Run it whenever you change the prompt, the model or the rules, and track two numbers:
- Missed escalations: conversations that needed a person but did not get one. These hurt customers.
- Unnecessary escalations: conversations the agent could have resolved. These cost your team time.
Make the handover seamless
A good handover feels like a colleague passing on a case, not a reset. The person picking it up should see:
- A two-line summary of what the customer wants.
- What the agent already tried and why it stopped.
- Relevant account details and any data the agent collected.
Tell the customer what happens next and how long it is likely to take. If no one is available, offer a callback or an email follow-up rather than leaving them waiting in a queue.
Measure success by resolved conversations and customer satisfaction, not by how few conversations reach your team.
Start strict, then relax
At launch, escalate generously. Review the handed-over conversations each week, find the ones the agent could safely have handled, and widen its scope one category at a time. This keeps quality high while the agent earns its autonomy.
