# SSCFormer: Push the Limit of Chunk-wise Conformer for Streaming ASR Using Sequentially Sampled Chunks and Chunked Causal Convolution

arXiv: https://arxiv.org/abs/2211.11419 · HTML: https://mlsanity.com/paper/2211.11419/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 29 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Recent Developments on ESPnet Toolkit Boosted by Conformer](https://mlsanity.com/paper/2010.13956/) (2020)
- [Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognition](https://mlsanity.com/paper/2010.10759/) (2020)
- [Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset](https://mlsanity.com/paper/2010.11395/) (2020)
- [Streaming automatic speech recognition with the transformer model](https://mlsanity.com/paper/2001.02674/) (2020)
- [On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition](https://mlsanity.com/paper/2005.14327/) (2020)
- [Squeezeformer: An Efficient Transformer for Automatic Speech Recognition](https://mlsanity.com/paper/2206.00888/) (2022)
- [WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2203.15455/) (2022)
- [Exploring Neural Transducers for End-to-End Speech Recognition](https://mlsanity.com/paper/1707.07413/) (2017)
- [Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition](https://mlsanity.com/paper/2012.05481/) (2020)
- [Streaming Transformer-based Acoustic Models Using Self-attention with Augmented Memory](https://mlsanity.com/paper/2005.08042/) (2020)
- [U2++: Unified Two-pass Bidirectional End-to-end Model for Speech Recognition](https://mlsanity.com/paper/2106.05642/) (2021)
- [Synchronous Transformers for End-to-End Speech Recognition](https://mlsanity.com/paper/1912.02958/) (2019)
- [Enhancing Monotonic Multihead Attention for Streaming ASR](https://mlsanity.com/paper/2005.09394/) (2020)
- [Advancing Acoustic-to-Word CTC Model](https://mlsanity.com/paper/1803.05566/) (2018)
- [Dual Causal/Non-Causal Self-Attention for Streaming End-to-End Speech Recognition](https://mlsanity.com/paper/2107.01269/) (2021)
- [Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognition](https://mlsanity.com/paper/2006.01712/) (2020)
- [CUSIDE: Chunking, Simulating Future Context and Decoding for Streaming ASR](https://mlsanity.com/paper/2203.16758/) (2022)
- [WNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognition](https://mlsanity.com/paper/2104.03587/) (2021)
- [Conformer with dual-mode chunked attention for joint online and offline ASR](https://mlsanity.com/paper/2206.11157/) (2022)
- [Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling](https://mlsanity.com/paper/2010.06030/) (2020)
- [Head-synchronous Decoding for Transformer-based Streaming ASR](https://mlsanity.com/paper/2104.12631/) (2021)
- [Shifted Chunk Encoder for Transformer Based Streaming End-to-End ASR](https://mlsanity.com/paper/2203.15206/) (2022)

## Built on by (most cited first, 1 of 1)

- [SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition](https://mlsanity.com/paper/2603.10005/) (2026)

## Abstract (arXiv)

Currently, the chunk-wise schemes are often used to make Automatic Speech Recognition (ASR) models to support streaming deployment. However, existing approaches are unable to capture the global context, lack support for parallel training, or exhibit quadratic complexity for the computation of multi-head self-attention (MHSA). On the other side, the causal convolution, no future context used, has become the de facto module in streaming Conformer. In this paper, we propose SSCFormer to push the limit of chunk-wise Conformer for streaming ASR using the following two techniques: 1) A novel cross-chunks context generation method, named Sequential Sampling Chunk (SSC) scheme, to re-partition chunks from regular partitioned chunks to facilitate efficient long-term contextual interaction within local chunks. 2)The Chunked Causal Convolution (C2Conv) is designed to concurrently capture the left context and chunk-wise future context. Evaluations on AISHELL-1 show that an End-to-End (E2E) CER 5.33% can achieve, which even outperforms a strong time-restricted baseline U2. Moreover, the chunk-wise MHSA computation in our model enables it to train with a large batch size and perform inference with linear complexity.