# Redwood: Using Collision Detection to Grow a Large-Scale Intent Classification Dataset

arXiv: https://arxiv.org/abs/2204.05483 · HTML: https://mlsanity.com/paper/2204.05483/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 12 Apr 2022.

## Builds on (most cited first, 16 of 16)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices](https://mlsanity.com/paper/2004.02984/) (2020)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Benchmarking Natural Language Understanding Services for building Conversational Agents](https://mlsanity.com/paper/1903.05566/) (2019)
- [MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark](https://mlsanity.com/paper/2008.09335/) (2020)
- [Talk2Car: Taking Control of Your Self-Driving Car](https://mlsanity.com/paper/1909.10838/) (2019)
- [End-to-End Slot Alignment and Recognition for Cross-Lingual NLU](https://mlsanity.com/paper/2004.14353/) (2020)
- [Adversarial Semantic Collisions](https://mlsanity.com/paper/2011.04743/) (2020)
- [Outlier Detection for Improved Data Quality and Diversity in Dialog Systems](https://mlsanity.com/paper/1904.03122/) (2019)
- [The Eighth Dialog System Technology Challenge](https://mlsanity.com/paper/1911.06394/) (2019)
- [TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition](https://mlsanity.com/paper/2110.15063/) (2021)
- [LSOIE: A Large-Scale Dataset for Supervised Open Information Extraction](https://mlsanity.com/paper/2101.11177/) (2021)
- [One Agent To Rule Them All: Towards Multi-agent Conversational AI](https://mlsanity.com/paper/2203.07665/) (2022)

## Built on by (most cited first, 6 of 6)

- [InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning](https://mlsanity.com/paper/2205.12673/) (2022)
- [Evaluating Out-of-Distribution Performance on Document Image Classifiers](https://mlsanity.com/paper/2210.07448/) (2022)
- [A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialog](https://mlsanity.com/paper/2207.13211/) (2022)
- [BlendX: Complex Multi-Intent Detection with Blended Patterns](https://mlsanity.com/paper/2403.18277/) (2024)
- [Generating Hard-Negative Out-of-Scope Data with ChatGPT for Intent Classification](https://mlsanity.com/paper/2403.05640/) (2024)
- [Improving End-to-End Speech Processing by Efficient Text Data Utilization with Latent Synthesis](https://mlsanity.com/paper/2310.05374/) (2023)

## Abstract (arXiv)

Dialog systems must be capable of incorporating new skills via updates over time in order to reflect new use cases or deployment scenarios. Similarly, developers of such ML-driven systems need to be able to add new training data to an already-existing dataset to support these new skills. In intent classification systems, problems can arise if training data for a new skill's intent overlaps semantically with an already-existing intent. We call such cases collisions. This paper introduces the task of intent collision detection between multiple datasets for the purposes of growing a system's skillset. We introduce several methods for detecting collisions, and evaluate our methods on real datasets that exhibit collisions. To highlight the need for intent collision detection, we show that model performance suffers if new data is added in such a way that does not arbitrate colliding intents. Finally, we use collision detection to construct and benchmark a new dataset, Redwood, which is composed of 451 ntent categories from 13 original intent classification datasets, making it the largest publicly available intent classification benchmark.