mlsanityData to 7 Oct 2026

Towards better decoding and language model integration in sequence to sequence models

arXiv 1612.02695 ↗First posted 8 Dec 2016cs.NE cs.CL cs.LGcited by 100

Not on the map. Why?

Abstract (arXiv)

The recently proposed Sequence-to-Sequence (seq2seq) framework advocates replacing complex data processing pipelines, such as an entire automatic speech recognition system, with a single neural network trained in an end-to-end fashion. In this contribution, we analyse an attention-based seq2seq speech recognition system that directly transcribes recordings into characters. We observe two shortcomings: overconfidence in its predictions and a tendency to produce incomplete transcriptions when language models are used. We propose practical solutions to both problems achieving competitive speaker independent word error rates on the Wall Street Journal dataset: without separate language models we reach 10.6% WER, while together with a trigram language model, we reach 6.7% WER.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 33; 2022: 26; 2023: 20; 2024: 12; 2025: 9.

100 papers.

Sort:

By area:

  1. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  2. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  3. 2021Unsupervised Speech Recognitioncited by 147
  4. 2023End-to-End Speech Recognition: A Surveycited by 127
  5. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
  6. 2023Stay on topic with Classifier-Free Guidancecited by 90
  7. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  8. 2021Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Studycited by 50
  9. 2023HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Modelscited by 41
  10. 2021Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognitioncited by 37
  11. 2025Dynamic Search for Inference-Time Alignment in Diffusion Modelscited by 36
  12. 2021Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognitioncited by 32
38 more
  1. 2022Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing?cited by 32
  2. 2021Spatially Varying Label Smoothing: Capturing Uncertainty from Expert Annotationscited by 31
  3. 2021On the limit of English conversational speech recognitioncited by 30
  4. 2021A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generationcited by 27
  5. 2022JOIST: A Joint Speech and Text Streaming Model For ASRcited by 24
  6. 2021Learning Word-Level Confidence For Subword End-to-End ASRcited by 22
  7. 2023An Overview on Language Models: Recent Developments and Outlookcited by 22
  8. 2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
  9. 2022Boundary Smoothing for Named Entity Recognitioncited by 21
  10. 2021Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognitioncited by 19
  11. 2021ASR Rescoring and Confidence Estimation with ELECTRAcited by 16
  12. 2023Large-scale Language Model Rescoring on Long-form Datacited by 14
  13. 2023CopyNE: Better Contextual ASR by Copying Named Entitiescited by 13
  14. 2022Improving Rare Word Recognition with LM-aware MWER Trainingcited by 12
  15. 2023Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attackscited by 12
  16. 2024Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxiacited by 12
  17. 2022Minimising Biasing Word Errors for Contextual ASR with the Tree-Constrained Pointer Generatorcited by 11
  18. 2022BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 11
  19. 2021Understanding Medical Conversations: Rich Transcription, Confidence Scores & Information Extractioncited by 10
  20. 2021Iterative Teaching by Label Synthesiscited by 10
  21. 2023Massively Multilingual Shallow Fusion with Large Language Modelscited by 10
  22. 2021Residual Energy-Based Models for End-to-End Speech Recognitioncited by 9
  23. 2022Internal Language Model Estimation Through Explicit Context Vector Learning for Attention-based Encoder-decoder ASRcited by 9
  24. 2023Contextual Biasing with the Knuth-Morris-Pratt Matching Algorithmcited by 9
  25. 2023Blank-regularized CTC for Frame Skipping in Neural Transducercited by 8
  26. 2024Label Smoothing Improves Machine Unlearningcited by 8
  27. 2024Large Language Models for Dysfluency Detection in Stuttered Speechcited by 8
  28. 2021Language model fusion for streaming end to end speech recognitioncited by 7
  29. 2022USTED: Improving ASR with a Unified Speech and Text Encoder-Decodercited by 7
  30. 2022Learning to segment fetal brain tissue from noisy annotationscited by 7
  31. 2021A Comparative Study on Neural Architectures and Training Methods for Japanese Speech Recognitioncited by 6
  32. 2022Distilling the Knowledge of BERT for CTC-based ASRcited by 6
  33. 2022Non-autoregressive Error Correction for CTC-based ASR with Phone-conditioned Masked LMcited by 6
  34. 2022A Likelihood Ratio based Domain Adaptation Method for E2E Modelscited by 5
  35. 2022Optimizing Bilingual Neural Transducer with Synthetic Code-switching Text Generationcited by 5
  36. 2022In and Out-of-Domain Text Adversarial Robustness via Label Smoothingcited by 5
  37. 2024Revisiting ASR Error Correction with Specialized Modelscited by 5
  38. 2025Whisper Has an Internal Word Alignercited by 5

50 most cited of 100 All 100 (JSON)