# Adapting Pretrained Transformer to Lattices for Spoken Language Understanding

arXiv: https://arxiv.org/abs/2011.00780 · HTML: https://mlsanity.com/paper/2011.00780/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2020.

## Built on by (most cited first, 12 of 12)

- ["How Robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversations](https://mlsanity.com/paper/2109.13489/) (2021)
- [N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypotheses](https://mlsanity.com/paper/2106.06519/) (2021)
- [Do as I mean, not as I say: Sequence Loss Training for Spoken Language Understanding](https://mlsanity.com/paper/2102.06750/) (2021)
- [FANS: Fusing ASR and NLU for on-device SLU](https://mlsanity.com/paper/2111.00400/) (2021)
- [Towards End-to-End Integration of Dialog History for Improved Spoken Language Understanding](https://mlsanity.com/paper/2204.05169/) (2022)
- [Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks](https://mlsanity.com/paper/2401.02921/) (2024)
- [ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding](https://mlsanity.com/paper/2311.11375/) (2023)
- [On joint training with interfaces for spoken language understanding](https://mlsanity.com/paper/2106.15919/) (2021)
- [Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding](https://mlsanity.com/paper/2205.00693/) (2022)
- [End-to-End Speech to Intent Prediction to improve E-commerce Customer Support Voicebot in Hindi and English](https://mlsanity.com/paper/2211.07710/) (2022)
- [Improving End-to-End Models for Set Prediction in Spoken Language Understanding](https://mlsanity.com/paper/2201.12105/) (2022)
- [Improve Text Classification Accuracy with Intent Information](https://mlsanity.com/paper/2212.07649/) (2022)

## Abstract (arXiv)

Lattices are compact representations that encode multiple hypotheses, such as speech recognition results or different word segmentations. It is shown that encoding lattices as opposed to 1-best results generated by automatic speech recognizer (ASR) boosts the performance of spoken language understanding (SLU). Recently, pretrained language models with the transformer architecture have achieved the state-of-the-art results on natural language understanding, but their ability of encoding lattices has not been explored. Therefore, this paper aims at adapting pretrained transformers to lattice inputs in order to perform understanding tasks specifically for spoken language. Our experiments on the benchmark ATIS dataset show that fine-tuning pretrained transformers with lattice inputs yields clear improvement over fine-tuning with 1-best results. Further evaluation demonstrates the effectiveness of our methods under different acoustic conditions. Our code is available at https://github.com/MiuLab/Lattice-SLU