I built my first artificial neural network powered chatbot in 2001 for my junior high school science fair. I spent my childhood fascinated by computers and I would type away long into the night to see what I could make a computer do. The most exciting goal was the chatbot—a program that could converse like a human.

I was incredibly excited as more and more news and discussion came out about language modeling and neural networks to build realistic chatbots and other programmers on the internet broke down the complex math so I could build one of my own. I hastily fed my creation the script of “Ferris Bueller’s Day Off” in hopes of evoking some of the titular character’s dry wit. When it had finished digesting I beamed when it started off a conversation with an inappropriate but grammatically correct statement. Unfortunately, while unintentionally hilarious, the balance of the chat was a non-grammatical soup of words.

However, nothing I built could compare to the systems that were actually winning competitions based on hundreds of thousands of hand-tuned rules. In the nearly two intervening decades it’s only been made more clear how far away we are from generating conversation without reliance on rules, even with the state of the art neural chat models. Despite all of the research available and decades upon decades of building up these conversational rules, the current record for the Alexa Prize challenge is under 10 minutes of sustained conversation.

The cost of keeping a rules-based system current and consistent is steep—and these systems are ineffective at handling anything more complex than basic interactions.

Joseph Hackman

Customer conversations are far more complex than small talk.

The further problem with business-oriented chatbots is that helping a customer through a problem is a different task than just making small-talk. Somehow, the bots need to learn all the details of the business and then learn to achieve the correct goals for the customer. Some of these, such as pricing details and promotions may change frequently. On current systems, that means spending time and money carefully curating responses and business information. Worse yet, the costs of keeping this data current and consistent scale exponentially. While this is incredible for consultants that reap thousands of billable hours, it’s not effective for businesses and their customers. The result seems to always be deploying half-finished products over budget or abandoning them entirely.

Machine learning informs human augmentation.

On the other hand, the successes of human augmentation are many. Siri, Alexa, and Google Assistant are terrible conversationalists, but fantastic at making their users more effective. I joined ASAPP on the promise of being able to apply this type of thinking to the business world. Rather than having subject matter experts or consultants maintain thousands of pages of carefully-written copy and logic in every language your business supports, we leverage the interactions agents have with customers. Every conversation between a customer service agent and a customer on the ASAPP platform provides thousands if not millions of opportunities for learning.

Continuous learning increases accuracy.

One simple example that comes to mind is that by noting what text the agents delete (and what they replace it with), our system learns a model of how agents edit their own messages in the specific context of the business. We then apply this data-driven model as agents type, and correct spelling mistakes automatically and with exceptionally high precision.

Even better, predictions of what an agent should say or do next turn out to be both feasible and effective. Although naturally requiring large and state-of-the-art models, no business rules or scripts are required, instead drawing all necessary information from the history of interactions had on the ASAPP platform. When an agent is interacting with our system, just by presenting suggestions of what types of responses are commonly used (or indeed even what types of personalized responses each specific agent uses) we can dramatically improve the response time.

I’ve always taken great pride in designing and building systems that can handle immense scale gracefully while serving customers with helpful AI technology. The systems we are building and designing here go beyond that aim: they adapt and improve ever faster the more they are used.