CCA-FChapters09

Chapter 09

Escalation & HITL

Rule-based triggers, the three that only look like triggers, and a handoff the operator can read cold.

D5guide part i

9.1

Escalate on rules, not on mood

Every reliable trigger is a fact about the request. None of them are facts about the customer's tone.

situationaction
"get me a manager"escalate immediately — do not attempt to solve first
policy is silent on the requestescalate — e.g. matching a competitor's price
no progress after reasonable attemptsescalate
amount over a thresholdescalate — enforced by a hook, not a prompt
multiple customers match the searchask for another identifier — never guess

9.1b

What is not a trigger

Three plausible-sounding mechanisms that the exam offers and that do not work.

Sentiment analysis

Mood does not correlate with case complexity. A calm customer can have an unresolvable case; an angry one can have a two-click fix.

Self-rated confidence 1–10

The model can be confidently wrong. Its calibration is poor, so the number does not track "I need help".

A trained classifier

Overengineering — and it needs labelled training data you probably do not have.

All three fail the same way: they measure something other than whether a human is needed.

9.2

Three escalation shapes

Which one applies depends on whether the human was asked for, and whether it was asked for twice.

customermessage"get me a manager"escalate_to_human — nowissue, no asktry to resolveescalate ifunresolvedfrustrationacknowledge,then offerescalate onreiteration
Dissatisfaction is not a manager request. Acknowledge, offer, and escalate only if they insist again.

9.3

The human sees only your summary

No transcript comes with it. Whatever is missing from the handoff object is missing, full stop.

identitycustomer_id, customer_name, order_id
issue_summaryone line the operator can read cold
root_causewhat actually went wrong, plus evidence (photos attached)
actions_takenevery tool call that mattered and its outcome — "offered a replacement, customer insists on refund"
recommended_actionyour call: "approve a full refund" of $89.99
escalation_reasonwhich trigger fired
Self-contained by construction: identity, what happened, what was tried, what you recommend.

9.4

Confidence routing, done properly

Per-field scores, thresholds tuned on labelled data, and audits that keep looking after the threshold is set.

extractionconfidence per fieldthresholdtuned on labelled dataautomatedhuman reviewstratified random sampling audits the automated lane too97% overall can hide 40% errors in one document typemeasure accuracy per type and per field, never only overall
High confidence plus stable measured accuracy earns automation. Everything else routes to a person.

Recall in 60 seconds

  1. Explicit request for a human → escalate immediately, no resolution attempt.
  2. Policy silent on the request → escalate. Multiple customer matches → ask for an identifier.
  3. Threshold amounts are enforced with a hook, not a prompt instruction.
  4. Sentiment, self-rated confidence and a trained classifier are all not escalation triggers.
  5. Confidence scores are valid for review routing, and a distractor for escalation.
  6. Dissatisfaction ≠ manager request: acknowledge → offer → escalate only on reiteration.
  7. The handoff object must stand alone — the operator never sees the transcript.
  8. Audit the automated lane by document type and field; an aggregate number hides pockets of failure.