# Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding

arXiv: https://arxiv.org/abs/2003.09831 · HTML: https://mlsanity.com/paper/2003.09831/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2020.

## Built on by (most cited first, 7 of 7)

- [What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data](https://mlsanity.com/paper/2106.06410/) (2021)
- [A Persian Benchmark for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/2303.00408/) (2023)
- [A BiRGAT Model for Multi-intent Spoken Language Understanding with Hierarchical Semantic Frames](https://mlsanity.com/paper/2402.18258/) (2024)
- [An Approach to Build Zero-Shot Slot-Filling System for Industry-Grade Conversational Assistants](https://mlsanity.com/paper/2406.08848/) (2024)
- [A survey of joint intent detection and slot-filling models in natural language understanding](https://mlsanity.com/paper/2101.08091/) (2021)
- [Multi-grained Label Refinement Network with Dependency Structures for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/2209.04156/) (2022)
- [Actively Discovering New Slots for Task-oriented Conversation](https://mlsanity.com/paper/2305.04049/) (2023)

## Abstract (arXiv)

Traditional slot filling in natural language understanding (NLU) predicts a one-hot vector for each word. This form of label representation lacks semantic correlation modelling, which leads to severe data sparsity problem, especially when adapting an NLU model to a new domain. To address this issue, a novel label embedding based slot filling framework is proposed in this paper. Here, distributed label embedding is constructed for each slot using prior knowledge. Three encoding methods are investigated to incorporate different kinds of prior knowledge about slots: atomic concepts, slot descriptions, and slot exemplars. The proposed label embeddings tend to share text patterns and reuses data with different slot labels. This makes it useful for adaptive NLU with limited data. Also, since label embedding is independent of NLU model, it is compatible with almost all deep learning based slot filling models. The proposed approaches are evaluated on three datasets. Experiments on single domain and domain adaptation tasks show that label embedding achieves significant performance improvement over traditional one-hot label representation as well as advanced zero-shot approaches.