How agents informed an AI-driven feature that saves 30 seconds of handle time
As the product manager for the agent experience on the ASAPP platform, I get many inputs to my team’s roadmap. In shaping our roadmap as a Product, Design and Research team we try to balance cutting-edge experimentation and data-driven advancements with ideas that come directly from talking to and observing users. We find enormous value in doing what we call side-by-sides with agents.
Agents are power users—they are adept with all the ins-and-outs of the product, which makes them incredible sources of insight into potential areas for growth. On a recent site visit we sat side by side with agents to get feedback on recent product updates and observe ongoing workday pain points. While we were there our team noticed an interesting behavior in how agents were composing messages.
Quick typing led to many typos—which some agents took the time to correct while others did not.
Whenever agents chose to type a message freehand, they would try to type quickly, sometimes on old, sticky keyboards, which led to lots of typos. After a typo, different agents reacted differently. Some would ignore the typos and send the message as is, favoring a quick response over grammatical correctness. Others would finish typing their full response and then spend time right clicking their typos which were underlined in red by Chrome. Many agents would notice typos midway through typing, stop, and backspace in order to manually correct their errors.
My team saw an opportunity to improve quality and efficiency. We’ve all had the experience of trying to get a thought out, mistyping, and feeling interrupted by the consideration of whether to stop and correct it, or just keep going and come back later. For agents, this is happening all day long as they respond to customers, slowing down their response time and distracting from the content of the message they’re composing.
Looking deeper into the data around misspellings, the team observed that agents tend to make the same mistakes over and over again. By simply focusing on the couple thousand most common typos, we could address the vast majority of typo occurrences. Instead of making agents manually correct the typo by right clicking in Chrome or manually retyping, why not just correct it as they go?
As we built our autocorrect feature we wanted to make sure to not over-zealously correct. We all hate when the iPhone corrects a word we meant to type right before we send a message. The team set a high bar– we would only consider the feature successful if agents undid less than 1% of the typos we corrected. When we released this capability in an A/B test, the results were staggering. Not only was the undo rate far lower than 1%, but the impact to response time and overall handle time was substantial. This feature alone was able to cut average handle time by as much as 30 seconds, not to mention raise the quality bar of the responses that agents were sending to customers.
Sometimes the biggest wins come from small simple changes. Making sure our team takes the time to sit with agents often, side by side, ensures we keep a close pulse on what changes will really make an impact, however simple.