# Handling Long-Tail Queries with Slice-Aware Conversational Systems

arXiv: https://arxiv.org/abs/2104.13216 · HTML: https://mlsanity.com/paper/2104.13216/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Weakly-supervised learning of visual relations](https://mlsanity.com/paper/1707.09472/) (2017)
- [Structured Prediction with Partial Labelling through the Infimum Loss](https://mlsanity.com/paper/2003.00920/) (2020)
- [Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices](https://mlsanity.com/paper/1909.06349/) (2019)
- [State-Regularized Recurrent Neural Networks](https://mlsanity.com/paper/1901.08817/) (2019)
- [Uncertainty Estimation and Calibration with Finite-State Probabilistic RNNs](https://mlsanity.com/paper/2011.12010/) (2020)
- [A scalable framework for learning from implicit user feedback to improve natural language understanding in large-scale conversational AI systems](https://mlsanity.com/paper/2010.12251/) (2020)
- [Neural model robustness for skill routing in large-scale conversational AI systems: A design choice exploration](https://mlsanity.com/paper/2103.03373/) (2021)
- [Slice-Aware Neural Ranking](https://mlsanity.com/paper/2010.03343/) (2020)

## Built on by (most cited first, 2 of 2)

- [Learning Slice-Aware Representations with Mixture of Attentions](https://mlsanity.com/paper/2106.02363/) (2021)
- [Data Augmentation for Improving Tail-traffic Robustness in Skill-routing for Dialogue Systems](https://mlsanity.com/paper/2306.04823/) (2023)

## Abstract (arXiv)

We have been witnessing the usefulness of conversational AI systems such as Siri and Alexa, directly impacting our daily lives. These systems normally rely on machine learning models evolving over time to provide quality user experience. However, the development and improvement of the models are challenging because they need to support both high (head) and low (tail) usage scenarios, requiring fine-grained modeling strategies for specific data subsets or slices. In this paper, we explore the recent concept of slice-based learning (SBL) (Chen et al., 2019) to improve our baseline conversational skill routing system on the tail yet critical query traffic. We first define a set of labeling functions to generate weak supervision data for the tail intents. We then extend the baseline model towards a slice-aware architecture, which monitors and improves the model performance on the selected tail intents. Applied to de-identified live traffic from a commercial conversational AI system, our experiments show that the slice-aware model is beneficial in improving model performance for the tail intents while maintaining the overall performance.