# TASU2: Controllable CTC Simulation for Alignment and Low-Resource Adaptation of Speech LLMs

arXiv: https://arxiv.org/abs/2604.08384 · HTML: https://mlsanity.com/paper/2604.08384/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 22 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 17 of 17)

- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs](https://mlsanity.com/paper/2407.04051/) (2024)
- [On The Landscape of Spoken Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.08528/) (2025)
- [A Survey on Speech Large Language Models for Understanding](https://mlsanity.com/paper/2410.18908/) (2024)
- [SlideSpeech: A Large-Scale Slide-Enriched Audio-Visual Corpus](https://mlsanity.com/paper/2309.05396/) (2023)
- [Improving Hybrid CTC/Attention End-to-end Speech Recognition with Pretrained Acoustic and Language Model](https://mlsanity.com/paper/2112.07254/) (2021)
- [Zero-shot Domain-sensitive Speech Recognition with Prompt-conditioning Fine-tuning](https://mlsanity.com/paper/2307.10274/) (2023)
- [AlignFormer: Modality Matching Can Achieve Better Zero-shot Instruction-Following Speech-LLM](https://mlsanity.com/paper/2412.01145/) (2024)
- [ASR data augmentation in low-resource settings using cross-lingual multi-speaker TTS and cross-lingual voice conversion](https://mlsanity.com/paper/2204.00618/) (2022)
- [Text-only adaptation in LLM-based ASR through text denoising](https://mlsanity.com/paper/2601.20900/) (2026)
- [Low-Resource Domain Adaptation for Speech LLMs via Text-Only Fine-Tuning](https://mlsanity.com/paper/2506.05671/) (2025)
- [Label-Synchronous Neural Transducer for Adaptable Online E2E Speech Recognition](https://mlsanity.com/paper/2311.11353/) (2023)
- [LegoSLM: Connecting LLM with Speech Encoder using CTC Posteriors](https://mlsanity.com/paper/2505.11352/) (2025)
- [Updating Only Encoders Prevents Catastrophic Forgetting of End-to-End ASR Models](https://mlsanity.com/paper/2207.00216/) (2022)
- [Blank Collapse: Compressing CTC emission for the faster decoding](https://mlsanity.com/paper/2210.17017/) (2022)
- [TASU: Text-Only Alignment for Speech Understanding](https://mlsanity.com/paper/2511.03310/) (2025)

## Abstract (arXiv)

Speech LLM post-training increasingly relies on efficient cross-modal alignment and robust low-resource adaptation, yet collecting large-scale audio-text pairs remains costly. Text-only alignment methods such as TASU reduce this burden by simulating CTC posteriors from transcripts, but they provide limited control over uncertainty and error rate, making curriculum design largely heuristic. We propose TASU2, a controllable CTC simulation framework that simulates CTC posterior distributions under a specified WER range, producing text-derived supervision that better matches the acoustic decoding interface. This enables principled post-training curricula that smoothly vary supervision difficulty without TTS. Across multiple source-to-target adaptation settings, TASU2 improves in-domain and out-of-domain recognition over TASU, and consistently outperforms strong baselines including text-only fine-tuning and TTS-based augmentation, while mitigating source-domain performance degradation.