CUSIDE-array: A Streaming Multi-Channel End-to-End Speech Recognition System with Realistic Evaluations
On the map: Speech and audio language models → Speaker recognition and diarization
Abstract (arXiv)
Recently multi-channel end-to-end (ME2E) ASR systems have emerged. While streaming single-channel end-to-end ASR has been extensively studied, streaming ME2E ASR is limited in exploration. Additionally, recent studies call attention to the gap between in-distribution (ID) and out-of-distribution (OOD) tests and doing realistic evaluations. This paper focuses on two research problems: realizing streaming ME2E ASR and improving OOD generalization. We propose the CUSIDE-array method, which integrates the recent CUSIDE methodology (Chunking, Simulating Future Context and Decoding) into the neural beamformer approach of ME2E ASR. It enables streaming processing of both front-end and back-end with a total latency of 402ms. The CUSIDE-array ME2E models are shown to achieve superior streaming results in both ID and OOD tests. Realistic evaluations confirm the advantage of CUSIDE-array in its capability to consume single-channel data to improve OOD generalization via back-end pre-training and ME2E fine-tuning.
Key facts. Cited by 1 paper in the map (data through 19 Oct 2026). Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models. Builds on 18 of its 32 references that are in the map. First posted on arXiv 13 Jul 2024.
Builds on
18 of 32 references.
Closest ancestors · 5
Same direction, newest first
- 2022MFCCA:Multi-Frame Cross-Channel attention for multi-speaker ASR in Multi-party meeting scenario11 Oct
- 2022VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition12 Sep
- 2022Exploiting Single-Channel Speech for Multi-Channel End-to-End Speech Recognition: A Comparative Study31 Mar
- 2021M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge14 Oct
- 2021Multi-Channel Transformer Transducer for Speech Recognition30 Aug
Other references · 13
- 2017Attention Is All You Needcited by 61,180
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,661
- 2015MUSAN: A Music, Speech, and Noise Corpuscited by 694
- 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 497
- 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 458
- 2020CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordingscited by 231
- 2017Pyroomacoustics: A Python package for audio room simulations and array processing algorithmscited by 175
- 2021AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenariocited by 125
- 2019Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hoppingcited by 43
- 2017Multichannel End-to-end Speech Recognitioncited by 24
- 2021Dual Causal/Non-Causal Self-Attention for Streaming End-to-End Speech Recognitioncited by 21
- 2022CUSIDE: Chunking, Simulating Future Context and Decoding for Streaming ASRcited by 20
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024CUSIDE-T: Chunking, Simulating Future and Decoding for Transducer based Streaming ASR
- 2021Exploiting Single-Channel Speech For Multi-channel End-to-end Speech Recognition
- 2024Joint Optimization of Streaming and Non-Streaming Automatic Speech Recognition with Multi-Decoder and Knowledge Distillation
- 2021Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-field Speech Recognition
- 2021Extremely Low Footprint End-to-End ASR System for Smart Device
- 2022Compute Cost Amortized Transformer for Streaming ASR
- 2022Streaming Target-Speaker ASR with Neural Transducer
- 2022SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale