# Augmenting Slot Values and Contexts for Spoken Language Understanding with Pretrained Models

arXiv: https://arxiv.org/abs/2108.08451 · HTML: https://mlsanity.com/paper/2108.08451/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Data Augmentation using Pre-trained Transformer Models](https://mlsanity.com/paper/2003.02245/) (2020)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [Sequence-to-Sequence Data Augmentation for Dialogue Language Understanding](https://mlsanity.com/paper/1807.01554/) (2018)
- [Data Augmentation for Spoken Language Understanding via Joint Variational Generation](https://mlsanity.com/paper/1809.02305/) (2018)
- [Is Your Goal-Oriented Dialog Model Performing Really Well? Empirical Analysis of System-wise Evaluation](https://mlsanity.com/paper/2005.07362/) (2020)
- [C2C-GenDA: Cluster-to-Cluster Generation for Data Augmentation of Slot Filling](https://mlsanity.com/paper/2012.07004/) (2020)
- [Data Augmentation for Spoken Language Understanding via Pretrained Language Models](https://mlsanity.com/paper/2004.13952/) (2020)
- [Joint Learning of Word and Label Embeddings for Sequence Labelling in Spoken Language Understanding](https://mlsanity.com/paper/1910.07150/) (2019)
- [Conditioned Query Generation for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1911.03698/) (2019)

## Built on by (most cited first, 1 of 1)

- [Contextual Data Augmentation for Task-Oriented Dialog Systems](https://mlsanity.com/paper/2310.10380/) (2023)

## Abstract (arXiv)

Spoken Language Understanding (SLU) is one essential step in building a dialogue system. Due to the expensive cost of obtaining the labeled data, SLU suffers from the data scarcity problem. Therefore, in this paper, we focus on data augmentation for slot filling task in SLU. To achieve that, we aim at generating more diverse data based on existing data. Specifically, we try to exploit the latent language knowledge from pretrained language models by finetuning them. We propose two strategies for finetuning process: value-based and context-based augmentation. Experimental results on two public SLU datasets have shown that compared with existing data augmentation methods, our proposed method can generate more diverse sentences and significantly improve the performance on SLU.