# RNN Transducer Models For Spoken Language Understanding

arXiv: https://arxiv.org/abs/2104.03842 · HTML: https://mlsanity.com/paper/2104.03842/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 18 of 18)

- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [Speech Model Pre-training for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/1904.03670/) (2019)
- [Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer](https://mlsanity.com/paper/1801.00841/) (2018)
- [Towards end-to-end spoken language understanding](https://mlsanity.com/paper/1802.08395/) (2018)
- [Improving RNN Transducer Modeling for End-to-End Speech Recognition](https://mlsanity.com/paper/1909.12415/) (2019)
- [From Audio to Semantics: Approaches to end-to-end spoken language understanding](https://mlsanity.com/paper/1809.09190/) (2018)
- [Joint Speech Recognition and Speaker Diarization via Sequence Transduction](https://mlsanity.com/paper/1907.05337/) (2019)
- [Advancing RNN Transducer Technology for Speech Recognition](https://mlsanity.com/paper/2103.09935/) (2021)
- [Leveraging Unpaired Text Data for Training End-to-End Speech-to-Intent Systems](https://mlsanity.com/paper/2010.04284/) (2020)
- [End-to-End Neural Transformer Based Spoken Language Understanding](https://mlsanity.com/paper/2008.10984/) (2020)
- [Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portability](https://mlsanity.com/paper/1906.07601/) (2019)
- [Using Speech Synthesis to Train End-to-End Spoken Language Understanding Models](https://mlsanity.com/paper/1910.09463/) (2019)
- [End-to-End Spoken Language Understanding Without Full Transcripts](https://mlsanity.com/paper/2009.14386/) (2020)
- [Guiding CTC Posterior Spike Timings for Improved Posterior Fusion and Knowledge Distillation](https://mlsanity.com/paper/1904.08311/) (2019)
- [Improving End-to-End Speech-to-Intent Classification with Reptile](https://mlsanity.com/paper/2008.01994/) (2020)
- [High quality, lightweight and adaptable TTS using LPCNet](https://mlsanity.com/paper/1905.00590/) (2019)
- [End-to-end architectures for ASR-free spoken language understanding](https://mlsanity.com/paper/1910.10599/) (2019)
- [Large-scale Transfer Learning for Low-resource Spoken Language Understanding](https://mlsanity.com/paper/2008.05671/) (2020)

## Built on by (most cited first, 7 of 7)

- [Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systems](https://mlsanity.com/paper/2203.00006/) (2022)
- [Integrating Dialog History into End-to-End Spoken Language Understanding Systems](https://mlsanity.com/paper/2108.08405/) (2021)
- [Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target](https://mlsanity.com/paper/2305.18096/) (2023)
- [On joint training with interfaces for spoken language understanding](https://mlsanity.com/paper/2106.15919/) (2021)
- [Improving End-to-End Models for Set Prediction in Spoken Language Understanding](https://mlsanity.com/paper/2201.12105/) (2022)
- [A new data augmentation method for intent classification enhancement and its application on spoken conversation datasets](https://mlsanity.com/paper/2202.10137/) (2022)
- [Improving End-to-End Speech Processing by Efficient Text Data Utilization with Latent Synthesis](https://mlsanity.com/paper/2310.05374/) (2023)

## Abstract (arXiv)

We present a comprehensive study on building and adapting RNN transducer (RNN-T) models for spoken language understanding(SLU). These end-to-end (E2E) models are constructed in three practical settings: a case where verbatim transcripts are available, a constrained case where the only available annotations are SLU labels and their values, and a more restrictive case where transcripts are available but not corresponding audio. We show how RNN-T SLU models can be developed starting from pre-trained automatic speech recognition (ASR) systems, followed by an SLU adaptation step. In settings where real audio data is not available, artificially synthesized speech is used to successfully adapt various SLU models. When evaluated on two SLU data sets, the ATIS corpus and a customer call center data set, the proposed models closely track the performance of other E2E models and achieve state-of-the-art results.