Blog

Every day contact center agents help millions of customers. To assist in each one of those contacts, an agent must first listen to the customer, diagnose the issue, and apply problem-solving to determine the correct sequence of actions to address customers’ needs. In each step of this process, agents must fetch, read, and update information from multiple back-office applications that, more often than not, are complex systems optimized to support business operations, not for providing a great user experience to agents.

A high learning curve

In practice, agents must know the purpose of each of those back-office systems to determine in which one they may find the information they seek. In addition, they must know the specific navigation flow that leads to the information within each system. Enabling this level of knowledge is expensive because it requires significant agent training, documentation and learning environments for trainees. And while training is a good start, experience is also a key. But, deep experience is rare in an industry where attrition averages 30-45% and can range to more than 100% annually. As a result, agents spend an appreciable amount of time wandering in applications looking for information. This leads to longer waiting times, incomplete answers, and more frustration for customers.

The limitations of RPA

In legacy systems, Robotic Process Automation (RPA) is a standard way of automating tasks in User Interfaces (UIs). However, RPA is highly resource-intensive since it requires manual scripting of each sequence of actions to be performed across many applications with potentially hundreds of navigation flows each. Hence, RPA simply doesn’t scale efficiently or effectively.

Where RPA is brittle and resource-intensive to scale, a machine learning system creates navigation flows automatically—and readily adapts when there are changes in the UI or agents’ behavior.

Nicolás D'Ippolito, PhD

Adding to the challenge, to automate tasks in RPA, developers must provide a list of actions to be performed. The definition of action in RPA requires a detailed description that allows the robot to identify the specific object in the UI to which it has to interact. As a result RPA scripts are highly coupled with the UI structure, making them very fragile to subtle changes in the UI. Although backend systems tend to change slowly, frontend systems often change frequently. Since the trend in the industry is to adopt web-based systems both internal and SaaS, fragility of RPA tools is an increasingly large problem.

Meeting the challenge with AI

In contrast, ASAPP AI-powered features can automatically determine the back-office system and the navigation flow that gets the agent to the required information. Our models evaluate the conversation context and identify potential navigation suggestions. When a recommendation is found, the agent is presented with a compact description of the system and flow leading to the required information. If the agent takes the recommendation, the ASAPP platform leads the agent to the information.

To implement these UI augmentation features we combine machine learning with stochastic analysis to generate behaviour models that abstract the potential interactions with the UI. The process to train this system is based on analysis of historical user interaction data. These models allow for efficient recall of high probability navigation paths towards a navigation goal from the current system state. This capability gives us the power to automatically create a robust navigation tool for any given application. We then combine our navigation tools with NLP and classification models to determine for each conversation context which navigation tool and flow to use.

In addition to the automation benefits, our UI augmentation greatly increases the resiliency to changes in the UI. Since we have a behavior model of every system we can detect deviations from standard usage patterns due to changes in navigation flows in the UI. We can also detect new states in the system that we didn’t observe before due to changes in the UI or agents’ usage patterns. In both cases, these deviations are considered during the automatic retraining cycles. Our models adapt to the changes in the UI, and navigation tools are re-generated, ensuring agent access to needed information is always current.

UI augmentation enables agents to spend less time navigating back-office systems and more time helping customers. This leads to faster issue resolution, and therefore, happier customers. In addition, reducing cognitive load for agents opens the possibility for digital agents to engage with more than one customer at a time. So more customers can be served with the same amount of agents, further increasing agent productivity while also improving customer satisfaction as their issues are addressed more quickly and accurately. With the power of machine learning, we can train UI augmentation features on any enterprise’s infrastructure, enabling those companies to quickly get the benefits of agent augmentation.

When should a customer experience agent upgrade or upsell a customer? Even for a talented sales agent, it can be difficult to answer this question—choosing the right timing and the right offer in the flow of the conversation whether over messaging or a phone call. Add a steady stream of interactions, this year’s unprecedented demand for support, and the imperative to minimize handle time and it becomes next to impossible.

Since inception we’ve been helping businesses solve crucial customer experience challenges like this one by developing cutting edge AI. Most recently, we were granted a patent for “Automated Upsells in Customer Conversations.” The promise of our technology hinges on those challenges that every CX agent is facing along with the massive scale at which our customers operate.

Using AI to predict when to upsell and what to offer can help consumer companies increase sales and grow revenue.

Shawn Henry, PhD

When agents are rushing from issue to issue, there’s often not enough time to access and contemplate the history and preferences of an individual customer. Yet we know that everything from a customer’s next billing date to their list of purchased products could be valuable predictors of their interest in a new product offering or upgrade. In fact, because our models can learn from millions or billions of related customer data points, we can both extract novel correlations and effectively leverage those insights in real time.

While many solutions in Customer Experience involve generic one-size-fits-all automations, our machine learning models consume a variety of customer specific data points to augment an expert human agent. Our model begins by breaking down the conversation (whether in text or voice) into the content so far, the topic(s) of conversation, and sentiment of the customer. This context is complemented by customer data, including metadata on their account, preferences, products etc. When deployed at contact centers that employ thousands, or tens of thousands of people, speech recognition, natural language processing and machine learning make it practical to determine what conversations are going to deliver higher likelihood of upsell success, even when customers call in for many different reasons and in different emotional states. Sometimes, for example, a customer may be very agitated, and an upsell attempt could not only degrade their experience but harm their perception of the brand, so our model may actually discourage an upsell in favor of maintaining and strengthening that relationship.

In addition to a customer’s own data, the product selection can be determined by interpolating that data with the product preferences of similar customers, as is done with collaborative filtering. All of these factors inform better predictions. Being able to help customers knowledgeably with new products, upgrades or offerings, regardless of department, can be done without friction as the ASAPP platform can automatically surface the right upsell opportunities at the right time to agents across a CX workforce to help grow revenues.

Regardless of industry and agnostic to the channels customers use to communicate, businesses can benefit by surfacing upsell opportunities automatically to agents. ASAPP’s patented technology can dramatically transform this process and accelerate revenue growth as well as a company’s journey to providing unparalleled customer experience.

Machine learning models offer a powerful way to predict properties of incoming messages such as sentiment, language, and intent based on previous examples. We can evaluate a model’s performance in multiple ways:

Classification error measures how often its predictions are correct.

Calibration error measures how closely the model’s confidence scores match the percentage of time the model is correct.

For example, if a model is correct 95% of the time, we’d say its classification error is 5%. If the same model always reports it is 99% sure its answers are correct, then its calibration error would be 4%. Together, these metrics help determine whether a model is accurate, inaccurate, overconfident, or underconfident.

Reducing both classification error and calibration error over time is crucial for integration into human workflows. It enables us to maximize customer impact in an iterative manner. For example, well-calibrated models can trigger mature platform features to use more automation only when they have a high chance of succeeding. Furthermore, proper calibration creates an intuitive scale on which to compare multiple models, so that the overall system always utilizes the most confident prediction and that this confidence matches how well the system will actually perform.

Automating workflows requires a high degree of confidence in the ML model predicting that this is what is needed. Proper calibration increases that confidence.

Ethan Elenberg, PhD

The models developed by ASAPP provide value to our customers not from raw predictions alone, but rather from how those predictions are incorporated into platform features for our users. Therefore, we take several steps throughout model development to understand, measure, and improve the accuracy of confidence measures.

For example, consider the difference between predicting 95% chance of rain versus 55% chance of rain. A meteorologist would recommend that viewers take an umbrella with them in the former case but might not in the latter case. This weather prediction analogy fits many of the ASAPP models used in intent classification and knowledge-base retrieval. If a model predicts “PAYBILL” with score 0.95, we can send the customer to the “Pay my bill” automated workflow with a high degree of confidence that this will serve their need. If the score is 0.55, we might want to disambiguate with the customer whether they wanted to “pay their bill” or do something else.

Following our intuition, we would like a model to return 0.95 when it is 95% accurate and 0.55 when it is only 55% accurate. Calibration enables us to achieve this alignment. Throughout model development, we track the mismatch between a model’s score and its empirical accuracy with a metric called expected calibration error (ECE). ASAPP models are designed with a method called temperature scaling, which adjusts their raw scores. This changes the average confidence level in a way that reduces calibration error while maintaining prediction accuracy. The results can be significant: For example, one of our temperature scaled models was shown to have 85% lower ECE than the original model.

When ASAPP incorporates AI technology into its products, we use model calibration as one of our main design criteria. This ensures that multiple machine learning models work together to create the best automated experience for our customers.

In Summary

Machine learning models can be either overconfident or underconfident in their predictions. The intent classification models developed by ASAPP are calibrated so that prediction scores match their expected accuracy—and deliver a high level of value to ASAPP customers.