# Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization

arXiv: https://arxiv.org/abs/2404.09956 · HTML: https://mlsanity.com/paper/2404.09956/

## Key facts

- Cited by 97 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 15 Apr 2024.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)
- LLM alignment / RLHF / preference optimisation / [Preference alignment for recommenders and RAG](https://mlsanity.com/alignment/preference-alignment-for-recommenders-and-rag/)

## Builds on (most cited first, 18 of 18)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training](https://mlsanity.com/paper/2108.06209/) (2021)
- [AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining](https://mlsanity.com/paper/2308.05734/) (2023)
- [Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models](https://mlsanity.com/paper/2301.12661/) (2023)
- [Efficient Diffusion Training via Min-SNR Weighting Strategy](https://mlsanity.com/paper/2303.09556/) (2023)
- [Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model](https://mlsanity.com/paper/2304.13731/) (2023)
- [Mustango: Toward Controllable Text-to-Music Generation](https://mlsanity.com/paper/2311.08355/) (2023)
- [Learning from Between-class Examples for Deep Sound Recognition](https://mlsanity.com/paper/1711.10282/) (2017)
- [Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation](https://mlsanity.com/paper/2109.05418/) (2021)
- [BATON: Aligning Text-to-Audio Model with Human Preference Feedback](https://mlsanity.com/paper/2402.00744/) (2024)

## Built on by (most cited first, 30 of 97)

- [MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis](https://mlsanity.com/paper/2412.15322/) (2024)
- [Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation](https://mlsanity.com/paper/2510.01284/) (2025)
- [TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization](https://mlsanity.com/paper/2412.21037/) (2024)
- [Video-Guided Foley Sound Generation with Multimodal Controls](https://mlsanity.com/paper/2411.17698/) (2024)
- [MOVA: Towards Scalable and Synchronized Video-Audio Generation](https://mlsanity.com/paper/2602.08794/) (2026)
- [Kling-Foley: Multimodal Diffusion Transformer for High-Quality Video-to-Audio Generation](https://mlsanity.com/paper/2506.19774/) (2025)
- [Online DPO: Online Direct Preference Optimization with Fast-Slow Chasing](https://mlsanity.com/paper/2406.05534/) (2024)
- [Align Anything: Training All-Modality Models to Follow Instructions with Language Feedback](https://mlsanity.com/paper/2412.15838/) (2024)
- [A Survey of Direct Preference Optimization](https://mlsanity.com/paper/2503.11701/) (2025)
- [Taming Data and Transformers for Audio Generation](https://mlsanity.com/paper/2406.19388/) (2024)
- [A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications](https://mlsanity.com/paper/2410.15595/) (2024)
- [AudioX: A Unified Framework for Anything-to-Audio Generation](https://mlsanity.com/paper/2503.10522/) (2025)
- [Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation](https://mlsanity.com/paper/2410.10676/) (2024)
- [LeVo: High-Quality Song Generation with Multi-Preference Alignment](https://mlsanity.com/paper/2506.07520/) (2025)
- [KAD: No More FAD! An Effective and Efficient Evaluation Metric for Audio Generation](https://mlsanity.com/paper/2502.15602/) (2025)
- [Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback](https://mlsanity.com/paper/2411.01834/) (2024)
- [AV-Link: Temporally-Aligned Diffusion Features for Cross-Modal Audio-Video Generation](https://mlsanity.com/paper/2412.15191/) (2024)
- [FlowSep: Language-Queried Sound Separation with Rectified Flow Matching](https://mlsanity.com/paper/2409.07614/) (2024)
- [Preference Alignment Improves Language Model-Based TTS](https://mlsanity.com/paper/2409.12403/) (2024)
- [Improving Text-To-Audio Models with Synthetic Captions](https://mlsanity.com/paper/2406.15487/) (2024)
- [FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation](https://mlsanity.com/paper/2410.12266/) (2024)
- [A Survey of Sustainability in Large Language Models: Applications, Economics, and Challenges](https://mlsanity.com/paper/2412.04782/) (2024)
- [Images that Sound: Composing Images and Sounds on a Single Canvas](https://mlsanity.com/paper/2405.12221/) (2024)
- [ETTA: Elucidating the Design Space of Text-to-Audio Models](https://mlsanity.com/paper/2412.19351/) (2024)
- [VinTAGe: Joint Video and Text Conditioning for Holistic Audio Generation](https://mlsanity.com/paper/2412.10768/) (2024)
- [SoPo: Text-to-Motion Generation Using Semi-Online Preference Optimization](https://mlsanity.com/paper/2412.05095/) (2024)
- [Zero-Shot Audio-Visual Editing via Cross-Modal Delta Denoising](https://mlsanity.com/paper/2503.20782/) (2025)
- [AudioEditor: A Training-Free Diffusion-Based Audio Editing Framework](https://mlsanity.com/paper/2409.12466/) (2024)
- [AudioGen-Omni: A Unified Multimodal Diffusion Transformer for Video-Synchronized Audio, Speech, and Song Generation](https://mlsanity.com/paper/2508.00733/) (2025)
- [Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing](https://mlsanity.com/paper/2604.10708/) (2026)

## Abstract (arXiv)

Generative multimodal content is increasingly prevalent in much of the content creation arena, as it has the potential to allow artists and media personnel to create pre-production mockups by quickly bringing their ideas to life. The generation of audio from text prompts is an important aspect of such processes in the music and film industry. Many of the recent diffusion-based text-to-audio models focus on training increasingly sophisticated diffusion models on a large set of datasets of prompt-audio pairs. These models do not explicitly focus on the presence of concepts or events and their temporal ordering in the output audio with respect to the input prompt. Our hypothesis is focusing on how these aspects of audio generation could improve audio generation performance in the presence of limited data. As such, in this work, using an existing text-to-audio model Tango, we synthetically create a preference dataset where each prompt has a winner audio output and some loser audio outputs for the diffusion model to learn from. The loser outputs, in theory, have some concepts from the prompt missing or in an incorrect order. We fine-tune the publicly available Tango text-to-audio model using diffusion-DPO (direct preference optimization) loss on our preference dataset and show that it leads to improved audio output over Tango and AudioLDM2, in terms of both automatic- and manual-evaluation metrics.