# Detecting Egregious Conversations between Customers and Virtual Agents

arXiv: https://arxiv.org/abs/1711.05780 · HTML: https://mlsanity.com/paper/1711.05780/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2017.

## Built on by (most cited first, 4 of 4)

- [Language Models in Dialogue: Conversational Maxims for Human-AI Interactions](https://mlsanity.com/paper/2403.15115/) (2024)
- [Towards Healthy AI: Large Language Models Need Therapists Too](https://mlsanity.com/paper/2304.00416/) (2023)
- [Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations](https://mlsanity.com/paper/2403.06009/) (2024)
- [What Went Wrong? Explaining Overall Dialogue Quality through Utterance-Level Impacts](https://mlsanity.com/paper/2111.00572/) (2021)

## Abstract (arXiv)

Virtual agents are becoming a prominent channel of interaction in customer service. Not all customer interactions are smooth, however, and some can become almost comically bad. In such instances, a human agent might need to step in and salvage the conversation. Detecting bad conversations is important since disappointing customer service may threaten customer loyalty and impact revenue. In this paper, we outline an approach to detecting such egregious conversations, using behavioral cues from the user, patterns in agent responses, and user-agent interaction. Using logs of two commercial systems, we show that using these features improves the detection F1-score by around 20% over using textual features alone. In addition, we show that those features are common across two quite different domains and, arguably, universal.