01
Voice assistant for customer support
Problem
The support IVR handles 6–7 million calls a year. It ran on a rule engine with static responses: callers got a fixed answer or an article link, and calls routed to the wrong expert were transferred again, wasting expert time.
Approach
Replaced it with an agentic system on LLMs and RAG, grounded in 10,000+ editor-maintained articles, hundreds of which change every day. It holds a real conversation: asks follow-ups, works through the caller's problem and answers from current content. When it can't resolve the issue, it captures the problem and its classification, then routes to one of 200+ expert queues using intent, subscription and product. Concept to production launch in under three months.
Key decisions
- Choosing the platform by evidence. Before settling on the voice platform (NLX), evaluated Amazon Nova Sonic and Google GECX too, scoring each on 20+ parameters: voice quality, latency, compatibility with the existing landscape, integration complexity, observability, support for ReAct-style interaction with models and the agent architecture around it, support for different models, the agent harness and guardrails.
- Grounding first. A RAG pipeline with hybrid search and reranking, so answers come from current articles and not from the model's memory.
- Chunking by experiment. Many chunking strategies, each scored against the retrieval evals, so the choice was measured and not assumed.
- Evaluation at every layer. Word error rate for speech-to-text and text-to-speech quality, retrieval and ranker metrics, Ragas for generation, and scores for completeness, helpfulness and accuracy. Speech errors flow into every downstream metric, so each layer had its own gate.
- Our own test bench, reused. That evaluation work led to an in-house call simulator and evaluator that scores calls across many parameters, built in a few weeks. A second team building voice support inside the product now uses the same evaluator, with audio arriving over RTC from the app instead of the phone.
- Guardrails for financial advice. The assistant works in a financial product, so what it may and may not say, and how it handles ambiguity, is documented and enforced with red-teaming exercises and automated checks.
- Latency as a requirement. Optimised the whole voice path: the model, prompt caching, article caching and pre-generated answers.
Impact
~28% → <20%
Transfer rate
15% → 7%
Speech-to-text word error rate
6–7M
Calls / yr