# Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize

arXiv: https://arxiv.org/abs/1812.10757 · HTML: https://mlsanity.com/paper/1812.10757/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Dec 2018.

## Built on by (most cited first, 29 of 29)

- [True Few-Shot Learning with Language Models](https://mlsanity.com/paper/2105.11447/) (2021)
- [Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations](https://mlsanity.com/paper/2308.11995/) (2023)
- [Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling](https://mlsanity.com/paper/2107.03451/) (2021)
- [Neural Approaches to Conversational Information Retrieval](https://mlsanity.com/paper/2201.05176/) (2022)
- [Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbots](https://mlsanity.com/paper/2209.03463/) (2022)
- [Alexa, Let's Work Together: Introducing the First Alexa Prize TaskBot Challenge on Conversational Task Assistance](https://mlsanity.com/paper/2209.06321/) (2022)
- [Understanding Multi-Turn Toxic Behaviors in Open-Domain Chatbots](https://mlsanity.com/paper/2307.09579/) (2023)
- [Leveraging Implicit Feedback from Deployment Data in Dialogue](https://mlsanity.com/paper/2307.14117/) (2023)
- [Interactive Evaluation of Dialog Track at DSTC9](https://mlsanity.com/paper/2207.14403/) (2022)
- [Don't Forget Your ABC's: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systems](https://mlsanity.com/paper/2212.09180/) (2022)
- [Multi-Sentence Knowledge Selection in Open-Domain Dialogue](https://mlsanity.com/paper/2203.00763/) (2022)
- [State-of-the-art in Open-domain Conversational AI: A Survey](https://mlsanity.com/paper/2205.00965/) (2022)
- [Improving Response Quality with Backward Reasoning in Open-domain Dialogue Systems](https://mlsanity.com/paper/2105.00079/) (2021)
- [Athena 2.0: Contextualized Dialogue Management for an Alexa Prize SocialBot](https://mlsanity.com/paper/2111.02519/) (2021)
- [Bootstrapping a User-Centered Task-Oriented Dialogue System](https://mlsanity.com/paper/2207.05223/) (2022)
- [Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent](https://mlsanity.com/paper/2207.12021/) (2022)
- [A Transformer-based Response Evaluator for Open-Domain Spoken Conversation](https://mlsanity.com/paper/2302.04424/) (2023)
- [Data Analytics for Smart cities: Challenges and Promises](https://mlsanity.com/paper/2109.05581/) (2021)
- [Data-Centric Improvements for Enhancing Multi-Modal Understanding in Spoken Conversation Modeling](https://mlsanity.com/paper/2412.15995/) (2024)
- [Viola: A Topic Agnostic Generate-and-Rank Dialogue System](https://mlsanity.com/paper/2108.11063/) (2021)
- [Dialoging Resonance: How Users Perceive, Reciprocate and React to Chatbot's Self-Disclosure in Conversational Recommendations](https://mlsanity.com/paper/2106.01666/) (2021)
- [Are Akpans Trick or Treat: Unveiling Helpful Biases in Assistant Systems](https://mlsanity.com/paper/2205.12554/) (2022)
- [Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems](https://mlsanity.com/paper/2212.09252/) (2022)
- [Score Before You Speak: Improving Persona Consistency in Dialogue Generation using Response Quality Scores](https://mlsanity.com/paper/2508.06886/) (2025)
- [Miutsu: NTU's TaskBot for the Alexa Prize](https://mlsanity.com/paper/2205.07446/) (2022)
- [The DialPort tools](https://mlsanity.com/paper/2208.10918/) (2022)
- [Let's Get Personal: Personal Questions Improve SocialBot Performance in the Alexa Prize](https://mlsanity.com/paper/2303.04953/) (2023)
- [Athena 2.0: Discourse and User Modeling in Open Domain Dialogue](https://mlsanity.com/paper/2308.01887/) (2023)
- [Amazon Nova AI Challenge -- Trusted AI: Advancing secure, AI-assisted software development](https://mlsanity.com/paper/2508.10108/) (2025)

## Abstract (arXiv)

Building open domain conversational systems that allow users to have engaging conversations on topics of their choice is a challenging task. Alexa Prize was launched in 2016 to tackle the problem of achieving natural, sustained, coherent and engaging open-domain dialogs. In the second iteration of the competition in 2018, university teams advanced the state of the art by using context in dialog models, leveraging knowledge graphs for language understanding, handling complex utterances, building statistical and hierarchical dialog managers, and leveraging model-driven signals from user responses. The 2018 competition also included the provision of a suite of tools and models to the competitors including the CoBot (conversational bot) toolkit, topic and dialog act detection models, conversation evaluators, and a sensitive content detection model so that the competing teams could focus on building knowledge-rich, coherent and engaging multi-turn dialog systems. This paper outlines the advances developed by the university teams as well as the Alexa Prize team to achieve the common goal of advancing the science of Conversational AI. We address several key open-ended problems such as conversational speech recognition, open domain natural language understanding, commonsense reasoning, statistical dialog management, and dialog evaluation. These collaborative efforts have driven improved experiences by Alexa users to an average rating of 3.61, the median duration of 2 mins 18 seconds, and average turns to 14.6, increases of 14%, 92%, 54% respectively since the launch of the 2018 competition. For conversational speech recognition, we have improved our relative Word Error Rate by 55% and our relative Entity Error Rate by 34% since the launch of the Alexa Prize. Socialbots improved in quality significantly more rapidly in 2018, in part due to the release of the CoBot toolkit.