# MGSC: A Multi-granularity Consistency Framework for Robust End-to-end Asr

arXiv: https://arxiv.org/abs/2508.15853 · HTML: https://mlsanity.com/paper/2508.15853/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 20 Aug 2025.

## Builds on (most cited first, 19 of 19)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Matcha-TTS: A fast TTS architecture with conditional flow matching](https://mlsanity.com/paper/2309.03199/) (2023)
- [Automatic Speech Recognition using Advanced Deep Learning Approaches: A survey](https://mlsanity.com/paper/2403.01255/) (2024)
- [CLNode: Curriculum Learning for Node Classification](https://mlsanity.com/paper/2206.07258/) (2022)
- [Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignment](https://mlsanity.com/paper/2406.17957/) (2024)
- [CR-CTC: Consistency regularization on CTC for improved speech recognition](https://mlsanity.com/paper/2410.05101/) (2024)
- [Improving Neural Biasing for Contextual Speech Recognition by Early Context Injection and Text Perturbation](https://mlsanity.com/paper/2407.10303/) (2024)
- [Contextualized Automatic Speech Recognition with Dynamic Vocabulary](https://mlsanity.com/paper/2405.13344/) (2024)
- [Self-Remixing: Unsupervised Speech Separation via Separation and Remixing](https://mlsanity.com/paper/2211.10194/) (2022)
- [Improving CTC-based ASR Models with Gated Interlayer Collaboration](https://mlsanity.com/paper/2205.12462/) (2022)
- [Supervised Attention in Sequence-to-Sequence Models for Speech Recognition](https://mlsanity.com/paper/2204.12308/) (2022)
- [Unsupervised Improvement of Audio-Text Cross-Modal Representations](https://mlsanity.com/paper/2305.01864/) (2023)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Heterogeneous Face Attribute Estimation](https://mlsanity.com/paper/2403.00561/) (2024)
- [Boosting CTC-Based ASR Using LLM-Based Intermediate Loss Regularization](https://mlsanity.com/paper/2506.22846/) (2025)
- [CitySeg: A 3D Open Vocabulary Semantic Segmentation Foundation Model in City-scale Scenarios](https://mlsanity.com/paper/2508.09470/) (2025)
- [Factual Consistency Oriented Speech Recognition](https://mlsanity.com/paper/2302.12369/) (2023)
- [Improved Contextual Recognition In Automatic Speech Recognition Systems By Semantic Lattice Rescoring](https://mlsanity.com/paper/2310.09680/) (2023)

## Abstract (arXiv)

End-to-end ASR models, despite their success on benchmarks, often pro-duce catastrophic semantic errors in noisy environments. We attribute this fragility to the prevailing 'direct mapping' objective, which solely penalizes final output errors while leaving the model's internal computational pro-cess unconstrained. To address this, we introduce the Multi-Granularity Soft Consistency (MGSC) framework, a model-agnostic, plug-and-play module that enforces internal self-consistency by simultaneously regulariz-ing macro-level sentence semantics and micro-level token alignment. Cru-cially, our work is the first to uncover a powerful synergy between these two consistency granularities: their joint optimization yields robustness gains that significantly surpass the sum of their individual contributions. On a public dataset, MGSC reduces the average Character Error Rate by a relative 8.7% across diverse noise conditions, primarily by preventing se-vere meaning-altering mistakes. Our work demonstrates that enforcing in-ternal consistency is a crucial step towards building more robust and trust-worthy AI.