Skip to main content
Customer-facing systems place agentic design under direct human scrutiny. Unlike code agents that iterate against objective tests, customer systems must satisfy subjective human expectations while handling the full variety of human communication. Unlike research systems that operate asynchronously, customer systems often require real-time response. And unlike internal tools where failures are inconvenient, customer system failures directly impact user experience and business outcomes. This chapter builds a customer service system that classifies inquiries, retrieves relevant knowledge, generates appropriate responses, and escalates when confidence is low. The system learns from resolutions to improve over time, accumulating expertise that makes it more capable with use.

14.1 The Customer Service Challenge

Customer service has properties that make it both suitable and challenging for automation. High volume, repetitive queries. Many customer inquiries follow patterns—password resets, order status, billing questions, feature explanations. These repetitive queries are ideal for automation because the system can learn effective responses and apply them consistently. Variable complexity. While many queries are straightforward, some are complex, ambiguous, or emotionally charged. A system must recognize when a query exceeds its capabilities and route to human agents appropriately. Quality expectations. Customers expect accurate information, appropriate tone, and genuine helpfulness. A response that’s technically correct but cold or confusing fails the customer even if it answers their question. Stakes matter. Stakes matter because poor customer service costs revenue, drives customers away, and can create legal liability when it spreads misinformation. The system must be reliable enough to trust with real customer interactions. These properties shape the design: classify to route appropriately, retrieve knowledge to ground responses, generate with appropriate tone, evaluate confidence to know when to escalate, and learn from outcomes to improve over time.

14.2 System Architecture

The customer system routes inquiries through specialized handlers based on classification, with escalation paths for uncertain or complex cases.
The Classifier analyzes incoming inquiries to determine intent and route to the appropriate handler. Classification happens quickly and determines the entire subsequent flow. Specialized Handlers process inquiries within their domain. Each handler has access to relevant tools, knowledge, and response patterns for its category. Confidence Checking evaluates whether the generated response should be delivered or escalated. In this chapter, confidence is a scalar between 0 and 1 computed by a separate evaluation prompt that scores relevance, accuracy, completeness, and tone, then aggregates these into an overall confidence score used for routing decisions. Low confidence triggers human review. Knowledge Base provides grounding for responses—FAQs, documentation, previous resolutions, and account information. The knowledge base is a combination of a vector index over FAQs and docs plus a keyed store for resolutions. Retrieval uses semantic search over this index and returns documents with a relevance score between 0 and 1.

14.3 Intent Classification

Classification is the routing decision that determines everything downstream. A misclassified inquiry goes to the wrong handler, uses wrong knowledge, and produces wrong responses.
Classification extracts multiple signals: the primary category, confidence level, sentiment, and urgency. These combine to inform routing decisions. A billing question from a frustrated, long-term customer gets different handling than a casual FAQ from a new user. This classifier is probabilistic and will mislabel some inquiries. In production, you should log disagreements between human agents and the classifier, then periodically retrain or adjust prompts based on those errors.

14.4 Knowledge-Grounded Response

Handlers generate responses grounded in retrieved knowledge. This prevents hallucination—responses cite actual documentation, policies, and facts rather than generating plausible-sounding fiction.
Response generation is explicitly grounded in retrieved knowledge. The system prompt instructs the model not to introduce facts that do not appear in the retrieved documents. The evaluation checks whether the response actually uses the provided knowledge and addresses the inquiry.

14.5 Confidence-Based Escalation

Not every inquiry should be handled automatically. The system must know when to escalate to human agents.
Escalation is multi-factor. Low confidence alone might not trigger escalation, but low confidence plus frustrated sentiment plus repeat contact definitely does. In a live system, escalation decisions must also respect queue capacity. You may cap simultaneous escalations or dynamically adjust thresholds when human queues are saturated, trading off speed against automation. The handoff summary ensures human agents have context to continue the conversation effectively.

14.6 Learning from Resolutions

The system increases its automated resolution rate by capturing successful human responses and reusing them as templates or retrieval items. When a human agent resolves an escalated ticket, or when customer feedback indicates satisfaction, the system captures what worked.
The patterns extracted here only matter if they feed back into the rest of the system. In practice, you would use getRelevantPatterns inside handlers like the FAQ handler by injecting the top patterns into the system prompt as additional examples or by storing them in the knowledge base so they are retrieved alongside documentation. This closes the loop between human resolutions and future automated answers.

14.7 The Complete Customer System

The following class wires the classifier, handlers, escalation manager, and learner into a single service entrypoint.

Key Takeaways

  • Customer systems require balancing automation with appropriate escalation
  • Classification determines routing—get it right and everything downstream works better
  • Response grounding in knowledge prevents hallucination and enables citation
  • Confidence-based escalation uses multiple signals, not just a single threshold
  • Tone adaptation based on sentiment and context improves customer experience
  • Learning from resolutions creates compounding improvement

Transition

Chapter 14 addressed human-facing interactions with emphasis on appropriate escalation. Chapter 15: Content Pipeline builds a creative system that operates autonomously while maintaining quality through review gates and performance feedback.