# Re-Sonance: A Dysarthric Asynchronous Real-Time Speech Conversion System Based on a Three-Stage Cascaded ASR-LLM-TTS Architecture

arXiv: https://arxiv.org/abs/2607.17615 · HTML: https://mlsanity.com/paper/2607.17615/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 11 of 11)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens](https://mlsanity.com/paper/2407.05407/) (2024)
- [Rambler: Supporting Writing With Speech via LLM-Assisted Gist Manipulation](https://mlsanity.com/paper/2401.10838/) (2024)
- [Convert and Speak: Zero-shot Accent Conversion with Minimum Supervision](https://mlsanity.com/paper/2408.10096/) (2024)
- [UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalization](https://mlsanity.com/paper/2401.14664/) (2024)
- [CDSD: Chinese Dysarthria Speech Database](https://mlsanity.com/paper/2310.15930/) (2023)
- [WESPER: Zero-shot and Realtime Whisper to Normal Voice Conversion for Whisper-based Speech Interactions](https://mlsanity.com/paper/2303.01639/) (2023)
- [PAVITS: Exploring Prosody-aware VITS for End-to-End Emotional Voice Conversion](https://mlsanity.com/paper/2403.01494/) (2024)
- [Decoupling Speaker-Independent Emotions for Voice Conversion Via Source-Filter Networks](https://mlsanity.com/paper/2110.01164/) (2021)
- [Assessment of Sign Language-Based versus Touch-Based Input for Deaf Users Interacting with Intelligent Personal Assistants](https://mlsanity.com/paper/2404.14605/) (2024)

## Abstract (arXiv)

Individuals with dysarthria face significant challenges in professional speaking scenarios such as conferences, presentations, and meetings, where real-time communication is crucial. While existing Augmentative and Alternative Communication (AAC) systems provide basic support, they often fail to meet the demands of professional speaking environments due to high latency and unnatural speech patterns. This paper presents Re-Sonance, a novel LLM-enhanced speech-driven AAC system designed for real-time professional speaking scenarios. By integrating Whisper ASR, Qwen LLM, and CosyVoice TTS, Re-Sonance achieves improved speech intelligibility and naturalness while maintaining real-time performance. Both subjective and objective evaluations using a Mandarin dysarthric speech dataset demonstrate that our speech reconstruction approach significantly improved intelligibility while preserving semantic coherence for speakers with mild to moderate dysarthria. Although performance remains limited for severe dysarthria cases, our findings validate the potential of LLM-based methods for enhancing speech-driven AAC systems, paving the way for more effective and accessible communication technologies.