mlsanityData to 14 Oct 2026

Variable Attention Masking for Configurable Transformer Transducer Speech Recognition

arXiv 2211.01438 ↗First posted 2 Nov 2022eess.AS cs.CL cs.SDcited by 14

Not on the map. Why?

Abstract (arXiv)

This work studies the use of attention masking in transformer transducer based speech recognition for building a single configurable model for different deployment scenarios. We present a comprehensive set of experiments comparing fixed masking, where the same attention mask is applied at every frame, with chunked masking, where the attention mask for each frame is determined by chunk boundaries, in terms of recognition accuracy and latency. We then explore the use of variable masking, where the attention masks are sampled from a target distribution at training time, to build models that can work in different configurations. Finally, we investigate how a single configurable model can be used to perform both first pass streaming recognition and second pass acoustic rescoring. Experiments show that chunked masking achieves a better accuracy vs latency trade-off compared to fixed masking, both with and without FastEmit. We also show that variable masking improves the accuracy by up to 8% relative in the acoustic re-scoring scenario.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 24 of its 32 references that are in the map. First posted on arXiv 2 Nov 2022.

Builds on

24 of 32 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  3. 2016Layer Normalizationcited by 5,669
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  6. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 235
  7. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  8. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
  9. 2020Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognitioncited by 113
  10. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 97
  11. 2020Streaming automatic speech recognition with the transformer modelcited by 73
  12. 2020FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularizationcited by 69
12 more
  1. 2020Cascaded encoders for unifying streaming and non-streaming ASRcited by 53
  2. 2021CAPE: Encoding Relative Positions with Continuous Augmented Positional Embeddingscited by 47
  3. 2019Transformer ASR with Contextual Block Processingcited by 28
  4. 2019Synchronous Transformers for End-to-End Speech Recognitioncited by 24
  5. 2020Transformer Transducer: One Model Unifying Streaming and Non-streaming Speech Recognitioncited by 24
  6. 2020Relative Positional Encoding for Speech Recognition and Direct Translationcited by 23
  7. 2020Conv-Transformer Transducer: Low Latency, Low Frame Rate, Streamable End-to-End Speech Recognitioncited by 23
  8. 2020Improving Streaming Automatic Speech Recognition With Non-Streaming Model Distillation On Unsupervised Datacited by 16
  9. 2021Reducing Streaming ASR Model Delay with Self Alignmentcited by 16
  10. 2021Dual Causal/Non-Causal Self-Attention for Streaming End-to-End Speech Recognitioncited by 14
  11. 2020Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognitioncited by 12
  12. 2022Streaming parallel transducer beam search with fast-slow cascaded encoderscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 3; 2025: 3; 2026: 3.

14 papers.

Sort:

By area:

  1. 2024Automatic Speech Recognition using Advanced Deep Learning Approaches: A surveycited by 50
  2. 2023Approximate Nearest Neighbour Phrase Mining for Contextual Speech Recognitioncited by 8
  3. 2023Chunked Attention-based Encoder-Decoder Model for Streaming Speech Recognitioncited by 7
  4. 2024Optimizing Contextual Speech Recognition Using Vector Quantization for Efficient Retrievalcited by 3
  5. 2023Cross-lingual Knowledge Transfer and Iterative Pseudo-labeling for Low-Resource Speech Recognition with Transducerscited by 2
  6. 2023Bigger is not Always Better: The Effect of Context Size on Speech Pre-Trainingcited by 2
  7. 2025Dynamic Context-Aware Streaming Pretrained Language Model For Inverse Text Normalizationcited by 2
  8. 2026Adaptive Turn-Taking for Real-time Multi-Party Voice Agentscited by 2
  9. 2024Fast Streaming Transducer ASR Prototyping via Knowledge Distillation with Whispercited by 1
  10. 2025Unifying Streaming and Non-streaming Zipformer-based ASRcited by 1
  11. 2023Neural Language Model Pruning for Automatic Speech Recognitioncited by 0
  12. 2025Segmental Attention Decoding With Long Form Acoustic Encodingscited by 0
2 more
  1. 2026Activation-Based Active Learning for In-Context Learning: Challenges and Insightscited by 0
  2. 2026Reading Less While Writing: A Closed-Form Bandwidth Dial for Streaming Multimodal Decoderscited by 0