# Analysis of Utterance Embeddings and Clustering Methods Related to Intent Induction for Task-Oriented Dialogue

arXiv: https://arxiv.org/abs/2212.02021 · HTML: https://mlsanity.com/paper/2212.02021/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 5 Dec 2022.

## Builds on (most cited first, 12 of 12)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings](https://mlsanity.com/paper/2204.10298/) (2022)
- [New Intent Discovery with Pre-training and Contrastive Learning](https://mlsanity.com/paper/2205.12914/) (2022)
- [Intent Mining from past conversations for conversational agent](https://mlsanity.com/paper/2005.11014/) (2020)
- [EASE: Entity-Aware Contrastive Learning of Sentence Embedding](https://mlsanity.com/paper/2205.04260/) (2022)
- [Dialog Intent Induction with Deep Multi-View Clustering](https://mlsanity.com/paper/1908.11487/) (2019)
- [Wasserstein K-means for clustering probability distributions](https://mlsanity.com/paper/2209.06975/) (2022)
- [Automatic Intent-Slot Induction for Dialogue Systems](https://mlsanity.com/paper/2103.08886/) (2021)
- [Variational Wasserstein Clustering](https://mlsanity.com/paper/1806.09045/) (2018)

## Built on by (most cited first, 1 of 1)

- [Unsupervised Flow Discovery from Task-oriented Dialogues](https://mlsanity.com/paper/2405.01403/) (2024)

## Abstract (arXiv)

The focus of this work is to investigate unsupervised approaches to overcome quintessential challenges in designing task-oriented dialog schema: assigning intent labels to each dialog turn (intent clustering) and generating a set of intents based on the intent clustering methods (intent induction). We postulate there are two salient factors for automatic induction of intents: (1) clustering algorithm for intent labeling and (2) user utterance embedding space. We compare existing off-the-shelf clustering models and embeddings based on DSTC11 evaluation. Our extensive experiments demonstrate that the combined selection of utterance embedding and clustering method in the intent induction task should be carefully considered. We also present that pretrained MiniLM with Agglomerative clustering shows significant improvement in NMI, ARI, F1, accuracy and example coverage in intent induction tasks. The source codes are available at https://github.com/Jeiyoon/dstc11-track2.