How AI monitors calls of 5,000+ sales representatives for actionable insights

Imagine trying to listen to and analyze sales calls of a 5,000+ strong sales force every single day.Sounds impossible, right?

That's exactly what the challenge was, where a 5,000+ strong sales force lives and breathes by the phone. The bread and butter was quite literally the conversations happening over these calls. But how to effectively monitor, understand, and improve such a massive volume of interactions?

This is where AI enters the picture, not as a futuristic concept, but as a practical, game-changing solution to a very real business problem.

The AI-Powered Pipeline: From Cacophony to Clarity

So, a thousand steps journey starts with the first step. How to tackle such a Herculean task? Let's break it down:

Call Recording via Telephony Service: All calls were routed to our telephony service and we leveraged their API to retrieve the audio files of all calls. Most telephony services such as Twilio, Exotel, Ozonetel, etc. provide APIs to retrieve the audio files of calls.

This is a crucial step as it forms the backbone of our system. To drive this in real-time as soon as the call ends, we registered a webhook with the telephony service. Setting up queues to manage call volumes in a concurrent manner was a major engineering challenge.
Text Transcription of Audio Files: Since, we wanted to leverage the cutting edge LLMs (such as Llama 3, Mistral, etc.), which could take in just text as input. We needed to convert all those spoken words into text. Here, we leveraged advanced Automatic Speech Recognition (ASR) technology, specifically OpenAI's Whisper which was a state-of-the-art open-source ASR model, to generate accurate transcripts of each call.

Since, the speakers often switched between various languages, the out-of-the-box ASR model was not 100% accurate. We had to build a custom error correction system to handle the errors in the ASR output.
Diarization: Speaker Identification: Raw transcripts are just the beginning. Each word in the transcript needs to be attributed to either the salesperson or the customer. Here, we employed additional tools to assess voice quality, diarize the output (fancy talk for figuring out who's speaking when) and time each spoken word to the exact second.

A major challenge that we faced here was when the voice quality of each speaker is similar or there was overlap in the spoken words of the salesperson and the customer, it becomes quite challenging to diarize the output. PyAnnote-Audio, which builds on top of Whisper for diarization, provided a robust solution to this problem.
LLM: The Intelligent Analyst: Now comes the most interesting part. These transcripts were appended to detailed prompts and fed into a Large Language Model (LLM) with carefully crafted instructions. The LLM's job? To extract crucial data points and provide them in a structured JSON format. The prompts were designed to extract information such as the salesperson's name, the customer's name, the purpose of the call, the key talking points, the outcome of the call, etc. Also, we made sure to make the output customizable so that the sales leaders can define the quality parameters as per the predefined quality parameters.
Quantifying Quality: The LLM didn't just summarize – it assigned numerical quality ratings based on various factors and parameters. This allowed us to objectively measure performance across multiple dimensions.
Emotional Intelligence: We didn't stop at just the words – we analyzed emotions too. How helpful was the salesperson? Did they demonstrate a solid understanding of our real estate projects?
Automated Flagging: Calls that fell below certain quality thresholds were automatically categorized for review by managers. No more hoping problematic calls would be caught by random sampling.
Performance Metrics at Scale: By analyzing the total number of monthly calls for each salesperson, along with their quality metrics, we created a comprehensive performance measurement system.
Customizable KPIs: We built in flexibility, allowing sales leaders to define both qualitative and quantitative parameters specific to their team's goals. The dashboard was built to be highly customizable so that the sales leaders can focus on the metrics that are important to them.

The Human Element: AI as an Enabler, Not a Replacement

It's crucial to note that this AI system isn't about replacing human judgment or micromanaging salespeople. Instead, it's about taking the heavy lifting off the sales managers so that they can focus on:

Empowering managers with data-driven insights
Identifying areas for targeted coaching and training
Recognizing and rewarding top performers
Spotting trends and opportunities across the entire sales organization

The system provides an unprecedented level of visibility, but it's still up to human leaders to interpret the data, make strategic decisions, and guide their teams to success.

Technical Challenges and Limitations

While we were able to build a production ready AI-powered monitoring system, it's not without a fair share of challenges. We try to list down some of the challenges that we faced to help

ASR Accuracy: While Whisper is impressive, accents, background noise, and technical jargon can still pose difficulties.
Emotional Analysis: Detecting nuanced emotional states remains an active area of research and improvement.
Data Privacy: Strict protocols are necessary to ensure customer information is protected throughout the process.
LLM Reliability: While powerful, LLMs can occasionally produce inconsistent results, requiring human oversight and quality checks.

The Road Ahead: Continuous Improvement

Our journey with AI-powered call monitoring is far from over. We're constantly refining the system, exploring new technologies, and finding ways to make it even more valuable to our sales teams and the organization as a whole. Some areas we're excited about:

Real-time analysis and coaching suggestions during live calls
Integration with CRM systems for a more holistic view of customer interactions
Predictive analytics to identify which leads are most likely to convert
Natural language generation to automate personalized sales call summaries

The future is AI-Powered Sales Performance Management

The days of relying solely on random call sampling or simplistic metrics are behind. AI has opened up possibilities for sales performance management that were unimaginable just a few years ago. By combining the analytical power of machines with human expertise and emotional intelligence, a new paradigm for sales excellence is being created.

For organizations dealing with high call volumes, the question is no longer "Can we afford to implement AI-powered monitoring?" but rather, "Can we afford not to?"

As we continue to push the boundaries of what's possible, one thing is clear: the future of sales performance management is here, and it's powered by AI.