mlsanityData to 7 Oct 2026

Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errors

arXiv 2310.16609 ↗First posted 25 Oct 2023cs.CL cs.AI cs.SDcited by 6

Not on the map. Why?

Abstract (arXiv)

In a spoken dialogue system, an NLU model is preceded by a speech recognition system that can deteriorate the performance of natural language understanding. This paper proposes a method for investigating the impact of speech recognition errors on the performance of natural language understanding models. The proposed method combines the back transcription procedure with a fine-grained technique for categorizing the errors that affect the performance of NLU models. The method relies on the usage of synthesized speech for NLU evaluation. We show that the use of synthesized speech in place of audio recording does not change the outcomes of the presented technique in a significant way.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 20 of its 35 references that are in the map. First posted on arXiv 25 Oct 2023.

Builds on

20 of 35 references.

References · 20

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  7. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  8. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  9. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  10. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
  11. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 142
  12. 2019RWTH ASR Systems for LibriSpeech: Hybrid vs Attention -- w/o Data Augmentationcited by 63
8 more
  1. 2017Listening while Speaking: Speech Chain by Deep Learningcited by 46
  2. 2020RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systemscited by 36
  3. 2020Robustness Testing of Language Understanding in Task-Oriented Dialogcited by 32
  4. 2020You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentationcited by 29
  5. 2018Back-Translation-Style Data Augmentation for End-to-End ASRcited by 28
  6. 2021fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkitcited by 24
  7. 2021On the Robustness of Intent Classification and Slot Labeling in Goal-oriented Dialog Systems to Real-world Noisecited by 9
  8. 2020Towards a Competitive End-to-End Speech Recognition for CHiME-6 Dinner Party Transcriptioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 3; 2026: 1.

6 papers.

  1. 2025reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputscited by 25
  2. 2024A Three-Pronged Approach to Cross-Lingual Adaptation with Multilingual LLMscited by 6
  3. 2025Measuring the Effect of Transcription Noise on Downstream Language Understanding Taskscited by 6
  4. 2023k* Distribution: Evaluating the Latent Space of Deep Neural Networks using Local Neighborhood Analysiscited by 2
  5. 2025Preservation of Language Understanding Capabilities in Speech-aware Large Language Modelscited by 1
  6. 2026WASIL: In-the-Wild Arabic Spoken Interactions with LLMscited by 1