# Self-Aware Feedback-Based Self-Learning in Large-Scale Conversational AI

arXiv: https://arxiv.org/abs/2205.00029 · HTML: https://mlsanity.com/paper/2205.00029/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 29 Apr 2022.

## Builds on (most cited first, 12 of 12)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](https://mlsanity.com/paper/1710.11214/) (2017)
- [Feedback Loop and Bias Amplification in Recommender Systems](https://mlsanity.com/paper/2007.13019/) (2020)
- [Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback](https://mlsanity.com/paper/1909.03601/) (2019)
- [Improving Multi-turn Dialogue Modelling with Utterance ReWriter](https://mlsanity.com/paper/1906.07004/) (2019)
- [Scaling Multi-Domain Dialogue State Tracking via Query Reformulation](https://mlsanity.com/paper/1903.05164/) (2019)
- [Feedback-Based Self-Learning in Large-Scale Conversational AI Agents](https://mlsanity.com/paper/1911.02557/) (2019)
- [Robust Dialogue Utterance Rewriting as Sequence Tagging](https://mlsanity.com/paper/2012.14535/) (2020)
- [Personalized Query Rewriting in Conversational AI Agents](https://mlsanity.com/paper/2011.04748/) (2020)
- [Mondegreen: A Post-Processing Solution to Speech Recognition Error Correction for Voice Search Queries](https://mlsanity.com/paper/2105.09930/) (2021)

## Built on by (most cited first, 3 of 3)

- [Reflective Linguistic Programming (RLP): A Stepping Stone in Socially-Aware AGI (SocialAGI)](https://mlsanity.com/paper/2305.12647/) (2023)
- [Benchmarking and Learning Real-World Customer Service Dialogue](https://mlsanity.com/paper/2510.22143/) (2025)
- [CURATRON: Complete and Robust Preference Data for Rigorous Alignment of Large Language Models](https://mlsanity.com/paper/2403.02745/) (2024)

## Abstract (arXiv)

Self-learning paradigms in large-scale conversational AI agents tend to leverage user feedback in bridging between what they say and what they mean. However, such learning, particularly in Markov-based query rewriting systems have far from addressed the impact of these models on future training where successive feedback is inevitably contingent on the rewrite itself, especially in a continually updating environment. In this paper, we explore the consequences of this inherent lack of self-awareness towards impairing the model performance, ultimately resulting in both Type I and II errors over time. To that end, we propose augmenting the Markov Graph construction with a superposition-based adjacency matrix. Here, our method leverages an induced stochasticity to reactively learn a locally-adaptive decision boundary based on the performance of the individual rewrites in a bi-variate beta setting. We also surface a data augmentation strategy that leverages template-based generation in abridging complex conversation hierarchies of dialogs so as to simplify the learning process. All in all, we demonstrate that our self-aware model improves the overall PR-AUC by 27.45%, achieves a relative defect reduction of up to 31.22%, and is able to adapt quicker to changes in global preferences across a large number of customers.