mlsanityData to 7 Oct 2026

Conversational AI: The Science Behind the Alexa Prize

arXiv 1801.03604 ↗First posted 11 Jan 2018cs.AI cs.CL cs.CYcited by 63

Not on the map. Why?

Abstract (arXiv)

Conversational agents are exploding in popularity. However, much work remains in the area of social conversation as well as free-form conversation over a broad range of domains and topics. To advance the state of the art in conversational AI, Amazon launched the Alexa Prize, a 2.5-million-dollar university competition where sixteen selected university teams were challenged to build conversational agents, known as socialbots, to converse coherently and engagingly with humans on popular topics such as Sports, Politics, Entertainment, Fashion and Technology for 20 minutes. The Alexa Prize offers the academic community a unique opportunity to perform research with a live system used by millions of users. The competition provided university teams with real user conversational data at scale, along with the user-provided ratings and feedback augmented with annotations by the Alexa team. This enabled teams to effectively iterate and make improvements throughout the competition while being evaluated in real-time through live user interactions. To build their socialbots, university teams combined state-of-the-art techniques with novel strategies in the areas of Natural Language Understanding, Context Modeling, Dialog Management, Response Generation, and Knowledge Acquisition. To support the efforts of participating teams, the Alexa Prize team made significant scientific and engineering investments to build and improve Conversational Speech Recognition, Topic Tracking, Dialog Evaluation, Voice User Experience, and tools for traffic management and scalability. This paper outlines the advances created by the university teams as well as the Alexa Prize team to achieve the common goal of solving the problem of Conversational AI.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 18; 2022: 20; 2023: 16; 2024: 3; 2025: 6.

63 papers.

Sort:

By area:

  1. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  2. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  3. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  4. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  5. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  6. 2022Capturing Failures of Large Language Models via Human Cognitive Biasescited by 97
  7. 2022When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgmentcited by 93
  8. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  9. 2022Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agentscited by 62
  10. 2021ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AIcited by 46
  11. 2022Neural Approaches to Conversational Information Retrievalcited by 41
  12. 2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
38 more
  1. 2021Revealing Persona Biases in Dialogue Systemscited by 33
  2. 2022Achieving Reliable Human Assessment of Open-Domain Dialogue Systemscited by 31
  3. 2023DialogBench: Evaluating LLMs as Human-like Dialogue Systemscited by 31
  4. 2022What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluationcited by 18
  5. 2021EmoNet: A Transfer Learning Framework for Multi-Corpus Speech Emotion Recognitioncited by 17
  6. 2022Mind the Gap! Injecting Commonsense Knowledge for Abstractive Dialogue Summarizationcited by 14
  7. 2023CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environmentscited by 14
  8. 2021Content Selection Network for Document-grounded Retrieval-based Chatbotscited by 10
  9. 2021Dompteur: Taming Audio Adversarial Examplescited by 10
  10. 2023Improving Open Language Models by Learning from Organic Interactionscited by 10
  11. 2021HERALD: An Annotation Efficient Method to Detect User Disengagement in Social Conversationscited by 9
  12. 2022Interactive Evaluation of Dialog Track at DSTC9cited by 9
  13. 2022Don't Forget Your ABC's: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systemscited by 9
  14. 2021TopicRefine: Joint Topic Prediction and Dialogue Response Generation for Multi-turn End-to-End Dialogue Systemcited by 5
  15. 2022Multi-Sentence Knowledge Selection in Open-Domain Dialoguecited by 5
  16. 2025Semantic Anchoring in Agentic Memory: Leveraging Linguistic Structures for Persistent Conversational Contextcited by 5
  17. 2021A Role-Selected Sharing Network for Joint Machine-Human Chatting Handoff and Service Satisfaction Analysiscited by 4
  18. 2022A Systematic Evaluation of Response Selection for Open Domain Dialoguecited by 4
  19. 2023Improving Open-Domain Dialogue Evaluation with a Causal Inference Modelcited by 4
  20. 2023The Open-domain Paradox for Chatbots: Common Ground as the Basis for Human-like Dialoguecited by 4
  21. 2023DialogVCS: Robust Natural Language Understanding in Dialogue System Upgradecited by 4
  22. 2021An animated picture says at least a thousand words: Selecting Gif-based Replies in Multimodal Dialogcited by 3
  23. 2021Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learningcited by 3
  24. 2022How "open" are the conversations with open-domain chatbots? A proposal for Speech Event based evaluationcited by 3
  25. 2022DialogQAE: N-to-N Question Answer Pair Extraction from Customer Service Chatlogcited by 3
  26. 2021WeaSuL: Weakly Supervised Dialogue Policy Learning: Reward Estimation for Multi-turn Dialoguecited by 2
  27. 2021DiscASP: A Graph-based ASP System for Finding Relevant Consistent Concepts with Applications to Conversational Socialbotscited by 2
  28. 2021Simple Dialogue System with AUDITEDcited by 2
  29. 2023Safer Conversational AI as a Source of User Delightcited by 2
  30. 2023ZRIGF: An Innovative Multimodal Framework for Zero-Resource Image-Grounded Dialogue Generationcited by 2
  31. 2022Are Akpans Trick or Treat: Unveiling Helpful Biases in Assistant Systemscited by 1
  32. 2023FCC: Fusing Conversation History and Candidate Provenance for Contextual Response Ranking in Dialogue Systemscited by 1
  33. 2023Rating Prediction in Conversational Task Assistants with Behavioral and Conversational-Flow Featurescited by 1
  34. 2023Social Commonsense-Guided Search Query Generation for Open-Domain Knowledge-Powered Conversationscited by 1
  35. 2024Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Surveycited by 1
  36. 2025KwaiChat: A Large-Scale Video-Driven Multilingual Mixed-Type Dialogue Corpuscited by 1
  37. 2021Sentiment Analysis for Open Domain Conversational Agentcited by 0
  38. 2021Improving Dialogue State Tracking by Joint Slot Modelingcited by 0

50 most cited of 63 All 63 (JSON)