mlsanityData to 14 Oct 2026

CR-CTC: Consistency regularization on CTC for improved speech recognition

arXiv 2410.05101 ↗First posted 7 Oct 2024eess.AS cs.LG cs.SDcited by 17

Not on the map. Why?

Abstract (arXiv)

Connectionist Temporal Classification (CTC) is a widely used method for automatic speech recognition (ASR), renowned for its simplicity and computational efficiency. However, it often falls short in recognition performance. In this work, we propose the Consistency-Regularized CTC (CR-CTC), which enforces consistency between two CTC distributions obtained from different augmented views of the input speech mel-spectrogram. We provide in-depth insights into its essential behaviors from three perspectives: 1) it conducts self-distillation between random pairs of sub-models that process different augmented views; 2) it learns contextual representation through masked prediction for positions within time-masked regions, especially when we increase the amount of time masking; 3) it suppresses the extremely peaky CTC distributions, thereby reducing overfitting and improving the generalization ability. Extensive experiments on LibriSpeech, Aishell-1, and GigaSpeech datasets demonstrate the effectiveness of our CR-CTC. It significantly improves the CTC performance, achieving state-of-the-art results comparable to those attained by transducer or systems combining CTC and attention-based encoder-decoder (CTC/AED). We release our code at https://github.com/k2-fsa/icefall.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 51 of its 69 references that are in the map. First posted on arXiv 7 Oct 2024.

Builds on

51 of 69 references.

References · 51

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  9. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  11. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  12. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
38 more
  1. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  4. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  5. 2016Deep Networks with Stochastic Depthcited by 979
  6. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
  7. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  8. 2018Born Again Neural Networkscited by 546
  9. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  10. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  11. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  12. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  13. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 340
  14. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 297
  15. 2021R-Drop: Regularized Dropout for Neural Networkscited by 202
  16. 2018Knowledge Distillation by On-the-Fly Native Ensemblecited by 186
  17. 2020Self-Distillation Amplifies Regularization in Hilbert Spacecited by 173
  18. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
  19. 2022Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Languagecited by 110
  20. 2020ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Contextcited by 97
  21. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 92
  22. 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 91
  23. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 80
  24. 2022WeNet 2.0: More Productive End-to-End Speech Recognition Toolkitcited by 47
  25. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
  26. 2021Momentum Pseudo-Labeling for Semi-Supervised Speech Recognitioncited by 40
  27. 2020Speech SIMCLR: Combining Contrastive and Reconstruction Objective for Self-supervised Speech Representation Learningcited by 34
  28. 2021Lhotse: a speech data representation library for the modern deep learning ecosystemcited by 33
  29. 2020Semi-Supervised Learning with Data Augmentation for End-to-End ASRcited by 19
  30. 2023Multi-Head State Space Model for Speech Recognitioncited by 14
  31. 2022Fast and parallel decoding for transducercited by 12
  32. 2024Less Peaky and More Accurate CTC Forced Alignment by Label Priorscited by 12
  33. 2022Co-training 2L Submodels for Visual Recognitioncited by 8
  34. 2023Masking meets Supervision: A Strong Learning Alliancecited by 3
  35. 2024Keep Decoding Parallel with Effective Knowledge Distillation from Language Models to End-to-end Speech Recogniserscited by 3
  36. 2022Improved Consistency Training for Semi-Supervised Sequence-to-Sequence ASR via Speech Chain Reconstruction and Self-Transcribingcited by 2
  37. 2022Contrastive Siamese Network for Semi-supervised Speech Recognitioncited by 2
  38. 2024Guiding Frame-Level CTC Alignments Using Self-knowledge Distillationcited by 2

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 11.

17 papers.

Sort:

By area:

  1. 2025ZIPA: A family of efficient models for multilingual phone recognitioncited by 19
  2. 2026An Empirical Recipe for Universal Phone Recognitioncited by 5
  3. 2026PRiSM: Benchmarking Phone Realization in Speech Modelscited by 4
  4. 2025HENT-SRT: Hierarchical Efficient Neural Transducer with Self-Distillation for Joint Speech Recognition and Translationcited by 1
  5. 2026Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMscited by 1
  6. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASRcited by 1
  7. 2026Reducing the Offline-Streaming Gap for Unified ASR Transducer with Consistency Regularizationcited by 1
  8. 2025A Differentiable Alignment Framework for Sequence-to-Sequence Modeling via Optimal Transportcited by 0
  9. 2025DESign: Dynamic Context-Aware Convolution and Efficient Subnet Regularization for Continuous Sign Language Recognitioncited by 0
  10. 2025MGSC: A Multi-granularity Consistency Framework for Robust End-to-end Asrcited by 0
  11. 2025Improving Audio Event Recognition with Consistency Regularizationcited by 0
  12. 2026Augmenting Dysarthric Speech Severity Assessment with MOS Supervisioncited by 0
5 more
  1. 2026DASH: Dual-View Self-Distillation with Multi-Layer Hidden Representations for Robust Speech Recognitioncited by 0
  2. 2026The Anatomy of the CTC Oracle Gap: Acoustic Exhaustion and Linguistic Recoverycited by 0
  3. 2026FUTO Swipe: Layout-Agnostic Neural Swipe Decodingcited by 0
  4. 2026On-Policy Self-Distillation for Multi-Dialect ASR: Mastering Dialects, Retaining Mandarincited by 0
  5. 2026BranchShine-CR: Compact Multilingual IPA Transcription with Self-Conditioned CTC and Consistency Regularizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023A CTC Alignment-based Non-autoregressive Transformer for End-to-end Automatic Speech Recognition
  2. 2023Improving Frame-level Classifier for Word Timings with Non-peaky CTC in End-to-End Automatic Speech Recognition
  3. 2023Self-distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach
  4. 2022Inter-KD: Intermediate Knowledge Distillation for CTC-Based Automatic Speech Recognition
  5. 2026CTC-Seeded Token Edit Refinement for Non-Autoregressive Speech Recognition
  6. 2026Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs
  7. 2022Peak-First CTC: Reducing the Peak Latency of CTC Models by Applying Peak-First Regularization
  8. 2022Blank Collapse: Compressing CTC emission for the faster decoding