mlsanityData to 7 Oct 2026

Learning a Dual-Mode Speech Recognition Model via Self-Pruning

arXiv 2207.11906 ↗First posted 25 Jul 2022eess.AS cs.CL cs.SDcited by 5

Not on the map. Why?

Abstract (arXiv)

There is growing interest in unifying the streaming and full-context automatic speech recognition (ASR) networks into a single end-to-end ASR model to simplify the model training and deployment for both use cases. While in real-world ASR applications, the streaming ASR models typically operate under more storage and computational constraints - e.g., on embedded devices - than any server-side full-context models. Motivated by the recent progress in Omni-sparsity supernet training, where multiple subnetworks are jointly optimized in one single model, this work aims to jointly learn a compact sparse on-device streaming ASR model, and a large dense server non-streaming model, in a single supernet. Next, we present that, performing supernet training on both wav2vec 2.0 self-supervised learning and supervised ASR fine-tuning can not only substantially improve the large non-streaming model as shown in prior works, and also be able to improve the compact sparse streaming model.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 28 of its 36 references that are in the map. First posted on arXiv 25 Jul 2022.

Builds on

28 of 36 references.

References · 28

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  7. 2018Self-Attention with Relative Position Representationscited by 1,112
  8. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  9. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
  10. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 185
  11. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 123
  12. 2020Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognitioncited by 111
16 more
  1. 2020A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latencycited by 99
  2. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 94
  3. 2019Transformer-based Acoustic Modeling for Hybrid Speech Recognitioncited by 86
  4. 2020A Better and Faster End-to-End Model for Streaming ASRcited by 68
  5. 2020On the Comparison of Popular End-to-End Models for Large Scale Speech Recognitioncited by 62
  6. 2020Cascaded encoders for unifying streaming and non-streaming ASRcited by 51
  7. 2019A comparison of end-to-end models for long-form speech recognitioncited by 42
  8. 2020Alignment Restricted Streaming Recurrent Neural Network Transducercited by 42
  9. 2020Improving RNN Transducer Based ASR with Auxiliary Taskscited by 27
  10. 2020Dynamic Sparsity Neural Networks for Automatic Speech Recognitioncited by 23
  11. 2021Omni-sparsity DNN: Fast Sparsity Optimization for On-Device Streaming E2E ASR via Supernetcited by 12
  12. 2020Benchmarking LF-MMI, CTC and RNN-T Criteria for Streaming ASRcited by 11
  13. 2022Conformer with dual-mode chunked attention for joint online and offline ASRcited by 9
  14. 2021Multi-mode Transformer Transducer with Stochastic Future Contextcited by 6
  15. 2021Streaming Transformer Transducer Based Speech Recognition Using Non-Causal Convolutioncited by 5
  16. 2021Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarioscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 0; 2025: 0; 2026: 1.

5 papers.

  1. 2022Learning ASR pathways: A sparse multilingual ASR modelcited by 12
  2. 2023TODM: Train Once Deploy Many Efficient Supernet-Based RNN-T Compression For On-device ASR Modelscited by 4
  3. 2023Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Modelcited by 3
  4. 2026Reducing the Offline-Streaming Gap for Unified ASR Transducer with Consistency Regularizationcited by 1
  5. 2023Knowledge Distillation from Non-streaming to Streaming ASR Encoder using Auxiliary Non-streaming Layercited by 0