# Improving CTC-AED model with integrated-CTC and auxiliary loss regularization

arXiv: https://arxiv.org/abs/2308.08449 · HTML: https://mlsanity.com/paper/2308.08449/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 34 references that are in the map.
- First posted on arXiv 15 Aug 2023.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Recent Developments on ESPnet Toolkit Boosted by Conformer](https://mlsanity.com/paper/2010.13956/) (2020)
- [WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2102.01547/) (2021)
- [A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency](https://mlsanity.com/paper/2003.12710/) (2020)
- [Intermediate Loss Regularization for CTC-based Speech Recognition](https://mlsanity.com/paper/2102.03216/) (2021)
- [Monotonic Chunkwise Attention](https://mlsanity.com/paper/1712.05382/) (2017)
- [Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predict](https://mlsanity.com/paper/2005.08700/) (2020)
- [A Better and Faster End-to-End Model for Streaming ASR](https://mlsanity.com/paper/2011.10798/) (2020)
- [Imputer: Sequence Modelling via Imputation and Dynamic Programming](https://mlsanity.com/paper/2002.08926/) (2020)
- [Two-Pass End-to-End Speech Recognition](https://mlsanity.com/paper/1908.10992/) (2019)
- [Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictions](https://mlsanity.com/paper/2104.02724/) (2021)
- [Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition](https://mlsanity.com/paper/2012.05481/) (2020)
- [Synchronous Transformers for End-to-End Speech Recognition](https://mlsanity.com/paper/1912.02958/) (2019)
- [Insertion-Based Modeling for End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2005.13211/) (2020)
- [FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularization](https://mlsanity.com/paper/2104.02882/) (2021)
- [Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognition](https://mlsanity.com/paper/2110.04891/) (2021)
- [SRU++: Pioneering Fast Recurrence with Attention for Speech Recognition](https://mlsanity.com/paper/2110.05571/) (2021)
- [Non-Autoregressive ASR with Self-Conditioned Folded Encoders](https://mlsanity.com/paper/2202.08474/) (2022)
- [One In A Hundred: Select The Best Predicted Sequence from Numerous Candidates for Streaming Speech Recognition](https://mlsanity.com/paper/2010.14791/) (2020)

## Abstract (arXiv)

Connectionist temporal classification (CTC) and attention-based encoder decoder (AED) joint training has been widely applied in automatic speech recognition (ASR). Unlike most hybrid models that separately calculate the CTC and AED losses, our proposed integrated-CTC utilizes the attention mechanism of AED to guide the output of CTC. In this paper, we employ two fusion methods, namely direct addition of logits (DAL) and preserving the maximum probability (PMP). We achieve dimensional consistency by adaptively affine transforming the attention results to match the dimensions of CTC. To accelerate model convergence and improve accuracy, we introduce auxiliary loss regularization for accelerated convergence. Experimental results demonstrate that the DAL method performs better in attention rescoring, while the PMP method excels in CTC prefix beam search and greedy search.