How do you set the right confidence threshold?

Start conservatively — set thresholds high so the AI escalates frequently. Review escalated conversations to see how many the AI could have handled correctly. Gradually lower thresholds in categories where the AI consistently handles escalated queries well, and keep thresholds high for high-stakes categories.

What happens to a conversation after confidence routing triggers?

Bookbag queues the conversation for a human agent with the full context and a note explaining why it was escalated. The customer receives a graceful message while they wait, rather than an abrupt transfer or a wrong answer.

Can confidence routing be bypassed?

Merchants can configure categories where the AI always escalates regardless of confidence — high-value customer accounts, sensitive billing disputes, or product safety issues where human judgment is always required. Confidence routing is a floor, not a ceiling.

Glossary

Confidence Routing

Confidence routing is the automated practice of evaluating an AI system's confidence score for each response and escalating the conversation to a human agent (or triggering a fallback behavior) when that score falls below a configured threshold — ensuring customers only receive AI responses the system is genuinely confident about.

Book a demo See pricing

What it means

Key insight

An AI that knows when to hand off is more valuable than one that answers everything — the willingness to escalate is what makes autonomous AI safe.

Confidence routing is the safety valve of an AI support deployment. Every AI response comes with an implicit or explicit confidence estimate: how certain is the system that its answer is accurate and appropriate for this query? When confidence is high — the intent is clear, a strong knowledge base match was found, the response is well-grounded — the AI answers autonomously. When confidence falls below threshold — the intent is ambiguous, retrieval returned weak matches, or the question falls outside the AI's scope — confidence routing triggers instead of guessing. The trigger can be a graceful fallback message ('Let me connect you with a team member who can help with this specifically'), an immediate human escalation, or a clarifying question. Well-calibrated confidence routing is what makes AI support safe to deploy autonomously: customers never receive a response the AI was actually uncertain about, and human agents receive only the conversations that genuinely need their judgment.

Why it matters

For Shopify merchants, the economic case for confidence routing is straightforward: a wrong AI answer costs more than a clean human escalation. Wrong answers generate follow-up contacts, negative reviews, and potential disputes. A clean escalation costs one agent interaction. Properly configured confidence thresholds mean the AI handles what it handles well (the large majority of standard queries) and escalates what it doesn't (edge cases, complex policy interpretations, frustrated or high-value customers) — delivering the best possible outcome for each tier of inquiry.

How Bookbag helps

Configurable Confidence Thresholds

Merchants set the confidence threshold for each intent category — lower thresholds for simple informational queries, higher thresholds for action-triggering responses like refunds — calibrating AI autonomy to match the stakes of each query type.

Soft vs. Hard Escalation

Bookbag supports both soft escalation (the AI acknowledges uncertainty and offers to connect a human while staying in the conversation) and hard escalation (immediate transfer to the human queue), configurable by scenario.

Confidence Score Transparency

In the agent dashboard, Bookbag shows the confidence scores behind escalated conversations so agents understand why the AI handed off and can calibrate thresholds based on real escalation patterns.

Frequently Asked Questions

See Bookbag in action

Join the ecommerce teams resolving more tickets, answering 24/7, and turning support into a revenue channel with Bookbag.