# DMP-TTS: Disentangled multi-modal Prompting for Controllable Text-to-Speech with Chained Guidance

arXiv: https://arxiv.org/abs/2512.09504 · HTML: https://mlsanity.com/paper/2512.09504/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2025.

## Built on by (most cited first, 3 of 3)

- [MM-Sonate: Multimodal Controllable Audio-Video Generation with Zero-Shot Voice Cloning](https://mlsanity.com/paper/2601.01568/) (2026)
- [UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions](https://mlsanity.com/paper/2604.22209/) (2026)
- [Joint Residual Reweighting for Classifier Free Guidance in Flow-Matching Zero-Shot TTS](https://mlsanity.com/paper/2606.25672/) (2026)

## Abstract (arXiv)

Controllable text-to-speech (TTS) systems face significant challenges in achieving independent manipulation of speaker timbre and speaking style, often suffering from entanglement between these attributes. We present DMP-TTS, a latent Diffusion Transformer (DiT) framework with explicit disentanglement and multi-modal prompting. A CLAP-based style encoder (Style-CLAP) aligns cues from reference audio and descriptive text in a shared space and is trained with contrastive learning plus multi-task supervision on style attributes. For fine-grained control during inference, we introduce chained classifier-free guidance (cCFG) trained with hierarchical condition dropout, enabling independent adjustment of content, timbre, and style guidance strengths. Additionally, we employ Representation Alignment (REPA) to distill acoustic-semantic features from a pretrained Whisper model into intermediate DiT representations, stabilizing training and accelerating convergence. Experiments show that DMP-TTS delivers stronger style controllability than open-source baselines while maintaining competitive intelligibility and naturalness. Code and demos will be available at https://y61329697.github.io/DMP-TTS/.