The situation
Putting an AI agent in front of consumer support at that scale means a wrong answer is not a one-person problem. Everyone agreed automation was the direction. Nobody had written down where its judgment ended.
The actual problem
The request was
“Turn on the AI agent and measure deflection.”
Discovery showed
Deflection rewards answering. Trust depends on knowing when not to answer. Without a rule for that, the agent optimizes toward confident wrong answers, which costs more than a queue does.
So I defined confidence thresholds per interaction class and wrote AI Operating Procedures that described the action, the review requirement, and the failure mode for each one. Automation expanded when the evidence cleared the bar, not when someone was ready to celebrate.
What I learned
- Modeled answer accuracy against confidence score to find where automation actually earned trust.
- Grouped interactions by consequence, from informational answers to account actions and safety topics.
- Sampled agent responses continuously rather than auditing after complaints arrived.
- Tracked escalation quality, since a bad handoff undoes a good answer.
How I framed it
Exhibit
AI Operating Procedures
- 01
Informational answer
- Threshold
- Moderate
- Human review
- Sampled
- Failure mode
- Low, correctable
- 02
Troubleshooting steps
- Threshold
- High
- Human review
- Sampled and flagged
- Failure mode
- Medium, wasted user time
- 03
Account or billing action
- Threshold
- Very high
- Human review
- Always
- Failure mode
- High, trust damage
- 04
Safety or legal topic
- Threshold
- Never automated
- Human review
- Human only
- Failure mode
- Severe
| Threshold | Human review | Failure mode | |
|---|---|---|---|
| 01Informational answer | Moderate | Sampled | Low, correctable |
| 02Troubleshooting steps | High | Sampled and flagged | Medium, wasted user time |
| 03Account or billing action | Very high | Always | High, trust damage |
| 04Safety or legal topic | Never automated | Human only | Severe |
Below threshold the agent does not guess. It educates or hands off with full context.
What we built
- 01Confidence thresholds tied to consequence rather than to a single global setting.
- 02Written AI Operating Procedures covering action, review, and escalation per interaction class.
- 03A warm handoff path that carries context so nobody repeats themselves to a human.
- 04Continuous sampling and quality review instead of complaint-driven auditing.
My role
- Authored
- Wrote the AI Operating Procedures, the confidence thresholds, the decision rules, the integration requirements, the escalation behavior, and the measurement model.
- Analysis
- Modeled accuracy against confidence to place each bar.
- Governance
- Set what evidence was required before raising any threshold.
- Operations
- Built the review cadence with the support team, not around them.
The outcome
- The AI agent became the primary consumer entry point with an explicit, defensible scope.
- Automation expanded on evidence, one interaction class at a time.
- Escalations arrived with context instead of restarting the conversation.
- The framework became the precursor to the autonomy levels I use in AI Ops work now.
What I'd do differently
Writing down what the AI is not allowed to do turned out to be the fastest way to expand what it was allowed to do. The rules made people comfortable enough to say yes.