# Improving Slot Filling by Utilizing Contextual Information

arXiv: https://arxiv.org/abs/1911.01680 · HTML: https://mlsanity.com/paper/1911.01680/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2019.

## Built on by (most cited first, 3 of 3)

- [AT-BERT: Adversarial Training BERT for Acronym Identification Winning Solution for SDU@AAAI-21](https://mlsanity.com/paper/2101.03700/) (2021)
- [Task Conditioned BERT for Joint Intent Detection and Slot-filling](https://mlsanity.com/paper/2308.06165/) (2023)
- [A survey of joint intent detection and slot-filling models in natural language understanding](https://mlsanity.com/paper/2101.08091/) (2021)

## Abstract (arXiv)

Slot Filling (SF) is one of the sub-tasks of Spoken Language Understanding (SLU) which aims to extract semantic constituents from a given natural language utterance. It is formulated as a sequence labeling task. Recently, it has been shown that contextual information is vital for this task. However, existing models employ contextual information in a restricted manner, e.g., using self-attention. Such methods fail to distinguish the effects of the context on the word representation and the word label. To address this issue, in this paper, we propose a novel method to incorporate the contextual information in two different levels, i.e., representation level and task-specific (i.e., label) level. Our extensive experiments on three benchmark datasets on SF show the effectiveness of our model leading to new state-of-the-art results on all three benchmark datasets for the task of SF.