# Example-Driven Intent Prediction with Observers

arXiv: https://arxiv.org/abs/2010.08684 · HTML: https://mlsanity.com/paper/2010.08684/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2020.

## Built on by (most cited first, 26 of 26)

- [Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning](https://mlsanity.com/paper/2109.06349/) (2021)
- [Multi-View Document Representation Learning for Open-Domain Dense Retrieval](https://mlsanity.com/paper/2203.08372/) (2022)
- [InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning](https://mlsanity.com/paper/2205.12673/) (2022)
- [Making LLMs Worth Every Penny: Resource-Limited Text Classification in Banking](https://mlsanity.com/paper/2311.06102/) (2023)
- [SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation](https://mlsanity.com/paper/2209.06664/) (2022)
- [InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation](https://mlsanity.com/paper/2112.01589/) (2021)
- [SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understanding](https://mlsanity.com/paper/2209.06638/) (2022)
- [ConvFiT: Conversational Fine-Tuning of Pretrained Language Models](https://mlsanity.com/paper/2109.10126/) (2021)
- [Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information](https://mlsanity.com/paper/2302.05096/) (2023)
- [LAD: Language Models as Data for Zero-Shot Dialog](https://mlsanity.com/paper/2207.14393/) (2022)
- [Breaking the Bank with ChatGPT: Few-Shot Text Classification for Finance](https://mlsanity.com/paper/2308.14634/) (2023)
- [NLU++: A Multi-Label, Slot-Rich, Generalisable Dataset for Natural Language Understanding in Task-Oriented Dialogue](https://mlsanity.com/paper/2204.13021/) (2022)
- [QAID: Question Answering Inspired Few-shot Intent Detection](https://mlsanity.com/paper/2303.01593/) (2023)
- [Learning to Select from Multiple Options](https://mlsanity.com/paper/2212.00301/) (2022)
- [Transfer-Free Data-Efficient Multilingual Slot Labeling](https://mlsanity.com/paper/2305.13528/) (2023)
- [Domain Adaptation in Intent Classification Systems: A Review](https://mlsanity.com/paper/2404.14415/) (2024)
- [Dense Retrieval as Indirect Supervision for Large-space Decision Making](https://mlsanity.com/paper/2310.18619/) (2023)
- [Confidence Calibration for Intent Detection via Hyperspherical Space and Rebalanced Accuracy-Uncertainty Loss](https://mlsanity.com/paper/2203.09278/) (2022)
- [The Devil is in the Details: On Models and Training Regimes for Few-Shot Intent Classification](https://mlsanity.com/paper/2210.06440/) (2022)
- [All Labels Together: Low-shot Intent Detection with an Efficient Label Semantic Encoding Paradigm](https://mlsanity.com/paper/2309.03563/) (2023)
- [RSVP: Customer Intent Detection via Agent Response Contrastive and Generative Pre-Training](https://mlsanity.com/paper/2310.09773/) (2023)
- [IntenDD: A Unified Contrastive Learning Approach for Intent Detection and Discovery](https://mlsanity.com/paper/2310.16761/) (2023)
- [TARDiS : Text Augmentation for Refining Diversity and Separability](https://mlsanity.com/paper/2501.02739/) (2025)
- [ESAinsTOD: A Unified End-to-End Schema-Aware Instruction-Tuning Framework for Task-Oriented Dialog Modeling](https://mlsanity.com/paper/2603.09691/) (2026)
- [Usage frequency and application variety of research methods in library and information science: Continuous investigation from 1991 to 2021](https://mlsanity.com/paper/2606.31081/) (2026)
- [Selecting Open-Weight Language Models for Zero-Shot Intent Classification: A Systematic Evaluation of 41 Models](https://mlsanity.com/paper/2607.27421/) (2026)

## Abstract (arXiv)

A key challenge of dialog systems research is to effectively and efficiently adapt to new domains. A scalable paradigm for adaptation necessitates the development of generalizable models that perform well in few-shot settings. In this paper, we focus on the intent classification problem which aims to identify user intents given utterances addressed to the dialog system. We propose two approaches for improving the generalizability of utterance classification models: (1) observers and (2) example-driven training. Prior work has shown that BERT-like models tend to attribute a significant amount of attention to the [CLS] token, which we hypothesize results in diluted representations. Observers are tokens that are not attended to, and are an alternative to the [CLS] token as a semantic representation of utterances. Example-driven training learns to classify utterances by comparing to examples, thereby using the underlying encoder as a sentence similarity model. These methods are complementary; improving the representation through observers allows the example-driven model to better measure sentence similarities. When combined, the proposed methods attain state-of-the-art results on three intent prediction datasets (textscbanking77, textscclinc150, textschwu64) in both the full data and few-shot (10 examples per intent) settings. Furthermore, we demonstrate that the proposed approach can transfer to new intents and across datasets without any additional training.