# A scalable framework for learning from implicit user feedback to improve natural language understanding in large-scale conversational AI systems

arXiv: https://arxiv.org/abs/2010.12251 · HTML: https://mlsanity.com/paper/2010.12251/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2020.

## Built on by (most cited first, 8 of 8)

- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Learning From Free-Text Human Feedback -- Collect New Datasets Or Extend Existing Ones?](https://mlsanity.com/paper/2310.15758/) (2023)
- [Scalable and Robust Self-Learning for Skill Routing in Large-Scale Conversational AI Systems](https://mlsanity.com/paper/2204.07135/) (2022)
- [Constrained Policy Optimization for Controlled Self-Learning in Conversational AI Systems](https://mlsanity.com/paper/2209.08429/) (2022)
- [Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI](https://mlsanity.com/paper/2412.15047/) (2024)
- [Handling Long-Tail Queries with Slice-Aware Conversational Systems](https://mlsanity.com/paper/2104.13216/) (2021)
- [Scalable and Safe Remediation of Defective Actions in Self-Learning Conversational Systems](https://mlsanity.com/paper/2305.10528/) (2023)
- [Demonstration of Pneuma-Seeker: Agentic System for Reifying and Fulfilling Information Needs on Tabular Data](https://mlsanity.com/paper/2604.14422/) (2026)

## Abstract (arXiv)

Natural Language Understanding (NLU) is an established component within a conversational AI or digital assistant system, and it is responsible for producing semantic understanding of a user request. We propose a scalable and automatic approach for improving NLU in a large-scale conversational AI system by leveraging implicit user feedback, with an insight that user interaction data and dialog context have rich information embedded from which user satisfaction and intention can be inferred. In particular, we propose a general domain-agnostic framework for curating new supervision data for improving NLU from live production traffic. With an extensive set of experiments, we show the results of applying the framework and improving NLU for a large-scale production system and show its impact across 10 domains.