mlsanityData to 7 Oct 2026

Why does CTC result in peaky behavior?

arXiv 2105.14849 ↗First posted 31 May 2021cs.LG cs.AI cs.CLcited by 32

Not on the map. Why?

Abstract (arXiv)

The peaky behavior of CTC models is well known experimentally. However, an understanding about why peaky behavior occurs is missing, and whether this is a good property. We provide a formal analysis of the peaky behavior and gradient descent convergence properties of the CTC loss and related training criteria. Our analysis provides a deep understanding why peaky behavior occurs and when it is suboptimal. On a simple example which should be trivial to learn for any model, we prove that a feed-forward neural network trained with CTC from uniform initialization converges towards peaky behavior with a 100% error rate. Our analysis further explains why CTC only works well together with the blank label. We further demonstrate that peaky behavior does not occur on other related losses including a label prior model, and that this improves convergence.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 4 of its 33 references that are in the map. First posted on arXiv 31 May 2021.

Builds on

4 of 33 references.

References · 4

  1. 2015EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decodingcited by 73
  2. 2016Wav2Letter: an End-to-End ConvNet-based Speech Recognition Systemcited by 43
  3. 2018RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognitioncited by 30
  4. 2015Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognitioncited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 7; 2024: 9; 2025: 5; 2026: 5.

32 papers.

Sort:

By area:

  1. 2023End-to-End Speech Recognition: A Surveycited by 127
  2. 2022Does Simultaneous Speech Translation need Simultaneous Models?cited by 22
  3. 2021CTC Variations Through New WFST Topologiescited by 17
  4. 2024Anatomy of Industrial Scale Multilingual ASRcited by 14
  5. 2022Bayes risk CTC: Controllable CTC alignment in Sequence-to-Sequence taskscited by 12
  6. 2024Less Peaky and More Accurate CTC Forced Alignment by Label Priorscited by 12
  7. 2023Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Modelcited by 9
  8. 2021On lattice-free boosted MMI training of HMM and CTC-based full-context ASR modelscited by 7
  9. 2024InterBiasing: Boost Unseen Word Recognition through Biasing Intermediate Predictionscited by 5
  10. 2024Enhancing Dysarthric Speech Recognition for Unseen Speakers via Prototype-Based Adaptationcited by 4
  11. 2025WCTC-Biasing: Retraining-free Contextual Biasing ASR with Wildcard CTC-based Keyword Spotting and Inter-layer Biasingcited by 3
  12. 2023Align With Purpose: Optimize Desired Properties in CTC Models with a General Plug-and-Play Frameworkcited by 2
20 more
  1. 2024The Conformer Encoder May Reverse the Time Dimensioncited by 2
  2. 2024HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASRcited by 2
  3. 2023Mask The Bias: Improving Domain-Adaptive Generalization of CTC-based ASR with Internal Language Model Estimationcited by 1
  4. 2023Improving Frame-level Classifier for Word Timings with Non-peaky CTC in End-to-End Automatic Speech Recognitioncited by 1
  5. 2023Revisiting the Entropy Semiring for Neural Speech Recognitioncited by 1
  6. 2025Segmentation-free Goodness of Pronunciationcited by 1
  7. 2026Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMscited by 1
  8. 2022Hear No Evil: Towards Adversarial Robustness of Automatic Speech Recognition via Multi-Task Learningcited by 0
  9. 2022Mining Word Boundaries in Speech as Naturally Annotated Word Segmentation Datacited by 0
  10. 2023Accurate and Reliable Confidence Estimation Based on Non-Autoregressive End-to-End Speech Recognition Systemcited by 0
  11. 2024Focused Discriminative Training For Streaming CTC-Trained Automatic Speech Recognition Modelscited by 0
  12. 2024Enhancing Code-Switching ASR Leveraging Non-Peaky CTC Loss and Deep Language Posterior Injectioncited by 0
  13. 2024Mining Word Boundaries from Speech-Text Parallel Data for Cross-domain Chinese Word Segmentationcited by 0
  14. 2025A Differentiable Alignment Framework for Sequence-to-Sequence Modeling via Optimal Transportcited by 0
  15. 2025Analyzing the Importance of Blank for CTC-Based Knowledge Distillationcited by 0
  16. 2025Audio-3DVG: Unified Audio -- Point Cloud Fusion for 3D Visual Groundingcited by 0
  17. 2026OpenFS: Multi-Hand-Capable Fingerspelling Recognition with Implicit Signing-Hand Detection and Frame-Wise Letter-Conditioned Synthesiscited by 0
  18. 2026Modeling Overlapped Speech with Shufflescited by 0
  19. 2026SMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spottingcited by 0
  20. 2026RAIDAL: Redundancy-Aware Information Density Active Learning for CTC-Based Continuous Sign Language Recognitioncited by 0