# Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systems arXiv: https://arxiv.org/abs/2203.00006 ยท HTML: https://mlsanity.com/paper/2203.00006/ ## Key facts - Cited by 20 papers in the map (data through 14 Oct 2026). - Builds on 19 of its 28 references that are in the map. - First posted on arXiv 26 Feb 2022. ## Builds on (most cited first, 19 of 19) - [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019) - [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018) - [Speech Model Pre-training for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/1904.03670/) (2019) - [Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer](https://mlsanity.com/paper/1801.00841/) (2018) - [Towards end-to-end spoken language understanding](https://mlsanity.com/paper/1802.08395/) (2018) - [Improving RNN Transducer Modeling for End-to-End Speech Recognition](https://mlsanity.com/paper/1909.12415/) (2019) - [From Audio to Semantics: Approaches to end-to-end spoken language understanding](https://mlsanity.com/paper/1809.09190/) (2018) - [Joint Speech Recognition and Speaker Diarization via Sequence Transduction](https://mlsanity.com/paper/1907.05337/) (2019) - [Advancing RNN Transducer Technology for Speech Recognition](https://mlsanity.com/paper/2103.09935/) (2021) - [Leveraging Unpaired Text Data for Training End-to-End Speech-to-Intent Systems](https://mlsanity.com/paper/2010.04284/) (2020) - [End-to-End Neural Transformer Based Spoken Language Understanding](https://mlsanity.com/paper/2008.10984/) (2020) - [Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portability](https://mlsanity.com/paper/1906.07601/) (2019) - [Using Speech Synthesis to Train End-to-End Spoken Language Understanding Models](https://mlsanity.com/paper/1910.09463/) (2019) - [End-to-End Spoken Language Understanding Without Full Transcripts](https://mlsanity.com/paper/2009.14386/) (2020) - [Improving End-to-End Speech-to-Intent Classification with Reptile](https://mlsanity.com/paper/2008.01994/) (2020) - [End-to-end architectures for ASR-free spoken language understanding](https://mlsanity.com/paper/1910.10599/) (2019) - [Large-scale Transfer Learning for Low-resource Spoken Language Understanding](https://mlsanity.com/paper/2008.05671/) (2020) - [HarperValleyBank: A Domain-Specific Spoken Dialog Corpus](https://mlsanity.com/paper/2010.13929/) (2020) - [RNN Transducer Models For Spoken Language Understanding](https://mlsanity.com/paper/2104.03842/) (2021) ## Built on by (most cited first, 20 of 20) - [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023) - [JOIST: A Joint Speech and Text Streaming Model For ASR](https://mlsanity.com/paper/2210.07353/) (2022) - [Massively Multilingual Shallow Fusion with Large Language Models](https://mlsanity.com/paper/2302.08917/) (2023) - [JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognition](https://mlsanity.com/paper/2302.08583/) (2023) - [Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target](https://mlsanity.com/paper/2305.18096/) (2023) - [Exploring Fine-Tuning of Large Audio Language Models for Spoken Language Understanding under Limited Speech Data](https://mlsanity.com/paper/2509.15389/) (2025) - [Zero-Shot End-to-End Spoken Language Understanding via Cross-Modal Selective Self-Training](https://mlsanity.com/paper/2305.12793/) (2023) - [Text-only Domain Adaptation using Unified Speech-Text Representation in Transducer](https://mlsanity.com/paper/2306.04076/) (2023) - [Improving Joint Speech-Text Representations Without Alignment](https://mlsanity.com/paper/2308.06125/) (2023) - [Effective Text Adaptation for LLM-based ASR through Soft Prompt Fine-Tuning](https://mlsanity.com/paper/2412.06967/) (2024) - [Text Injection for Neural Contextual Biasing](https://mlsanity.com/paper/2406.02921/) (2024) - [Cascaded Cross-Modal Transformer for Audio-Textual Classification](https://mlsanity.com/paper/2401.07575/) (2024) - [ASTRA: Aligning Speech and Text Representations for Asr without Sampling](https://mlsanity.com/paper/2406.06664/) (2024) - [EM-Network: Oracle Guided Self-distillation for Sequence Learning](https://mlsanity.com/paper/2306.10058/) (2023) - [Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Tracking](https://mlsanity.com/paper/2312.01842/) (2023) - [Joint Speech and Text Training for LLM-Based End-to-End Spoken Dialogue State Tracking](https://mlsanity.com/paper/2511.22503/) (2025) - [End-to-End Open Vocabulary Keyword Search With Multilingual Neural Representations](https://mlsanity.com/paper/2308.08027/) (2023) - [Text-Only Domain Adaptation for End-to-End Speech Recognition through Down-Sampling Acoustic Representation](https://mlsanity.com/paper/2309.02459/) (2023) - [Written Term Detection Improves Spoken Term Detection](https://mlsanity.com/paper/2407.04601/) (2024) - [Text-Utilization for Encoder-dominated Speech Recognition Models](https://mlsanity.com/paper/2604.26514/) (2026) ## Abstract (arXiv) The lack of speech data annotated with labels required for spoken language understanding (SLU) is often a major hurdle in building end-to-end (E2E) systems that can directly process speech inputs. In contrast, large amounts of text data with suitable labels are usually available. In this paper, we propose a novel text representation and training methodology that allows E2E SLU systems to be effectively constructed using these text resources. With very limited amounts of additional speech, we show that these models can be further improved to perform at levels close to similar systems built on the full speech datasets. The efficacy of our proposed approach is demonstrated on both intent and entity tasks using three different SLU datasets. With text-only training, the proposed system achieves up to 90% of the performance possible with full speech training. With just an additional 10% of speech data, these models significantly improve further to 97% of full performance.