mlsanityData to 7 Oct 2026

TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization

arXiv 2412.21037 ↗First posted 30 Dec 2024cs.SD cs.AI cs.CLcited by 67

On the map: Diffusion models → Audio, speech and music

also in 1 more
Abstract (arXiv)

We introduce TangoFlux, an efficient Text-to-Audio (TTA) generative model with 515M parameters, capable of generating up to 30 seconds of 44.1kHz audio in just 3.7 seconds on a single A40 GPU. A key challenge in aligning TTA models lies in the difficulty of creating preference pairs, as TTA lacks structured mechanisms like verifiable rewards or gold-standard answers available for Large Language Models (LLMs). To address this, we propose CLAP-Ranked Preference Optimization (CRPO), a novel framework that iteratively generates and optimizes preference data to enhance TTA alignment. We demonstrate that the audio preference dataset generated using CRPO outperforms existing alternatives. With this framework, TangoFlux achieves state-of-the-art performance across both objective and subjective benchmarks. We open source all code and models to support further research in TTA generation.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 43 of its 54 references that are in the map. First posted on arXiv 30 Dec 2024.

Builds on

43 of 54 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation16 Oct
  2. 2024Long-form music generation with latent diffusion16 Apr
  3. 2024Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization15 Apr
  4. 2024Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation2 Jan
  5. 2023Generative Pre-training for Speech with Flow Matching25 Oct
  6. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model24 Apr
  7. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models30 Jan
  8. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Models29 Jan

Other references · 35

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
23 more
  1. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  3. 2016Improved Techniques for Training GANscited by 2,938
  4. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  5. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  6. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  7. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  8. 2024Self-Rewarding Language Modelscited by 654
  9. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  10. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
  11. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  12. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  13. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 266
  14. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 242
  15. 2024From r to Q^*: Your Language Model is Secretly a Q-Functioncited by 242
  16. 2024Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positivecited by 239
  17. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 178
  18. 2020OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transportcited by 143
  19. 2024Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithmscited by 139
  20. 2024sDPO: Don't Use Your Data All at Oncecited by 46
  21. 2024Improving Text-To-Audio Models with Synthetic Captionscited by 19
  22. 2024BATON: Aligning Text-to-Audio Model with Human Preference Feedbackcited by 18
  23. 2024Adaptive Non-uniform Timestep Sampling for Accelerating Diffusion Model Trainingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 28; 2026: 39.

67 papers.

Sort:

By area:

  1. 2025DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusioncited by 46
  2. 2025HunyuanVideo-Foley: Multimodal Diffusion with Representation Alignment for High-Fidelity Foley Audio Generationcited by 42
  3. 2026MOVA: Towards Scalable and Synchronized Video-Audio Generationcited by 39
  4. 2026JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generationcited by 22
  5. 2025FreeAudio: Training-Free Timing Planning for Controllable Long-Form Text-to-Audio Generationcited by 21
  6. 2025MeanAudio: Fast and Faithful Text-to-Audio Generation with Mean Flowscited by 14
  7. 2025EnvSDD: Benchmarking Environmental Sound Deepfake Detectioncited by 11
  8. 2025UALM: Unified Audio Language Model for Understanding, Generation and Reasoningcited by 10
  9. 2025JAM: A Tiny Flow-based Song Generator with Fine-grained Controllability and Aesthetic Alignmentcited by 9
  10. 2025AnimeShooter: A Multi-Shot Animation Dataset for Reference-Guided Video Generationcited by 8
  11. 2026Apollo: Unified Multi-Task Audio-Video Joint Generationcited by 8
  12. 2026AQAScore: Evaluating Semantic Alignment in Text-to-Audio Generation via Audio Question Answeringcited by 8
38 more
  1. 2025Language Model Based Text-to-Audio Generation: Anti-Causally Aligned Collaborative Residual Transformerscited by 7
  2. 2025SonicMaster: Towards Controllable All-in-One Music Restoration and Masteringcited by 6
  3. 2025AudioMoG: Guiding Audio Generation with Mixture-of-Guidancecited by 5
  4. 2025MR-FlowDPO: Multi-Reward Direct Preference Optimization for Flow-Matching Text-to-Music Generationcited by 5
  5. 2026Stable Audio 3cited by 5
  6. 2025DRAGON: Distributional Rewards Optimize Diffusion Generative Modelscited by 4
  7. 2025FlexSED: Towards Open-Vocabulary Sound Event Detectioncited by 4
  8. 2026Bagpiper: Solving Open-Ended Audio Tasks via Rich Captionscited by 4
  9. 2026Woosh: A Sound Effects Foundation Modelcited by 4
  10. 2025AudioStory: Generating Long-Form Narrative Audio with Large Language Modelscited by 3
  11. 2025RFM-Editing: Rectified Flow Matching for Text-guided Audio Editingcited by 3
  12. 2025Foley Control: Aligning a Frozen Latent Text-to-Audio Model to Videocited by 3
  13. 2025Hear What Matters! Text-conditioned Selective Video-to-Audio Generationcited by 3
  14. 2026TuneJury: An Open Metric for Improving Music Generation Preference Alignmentcited by 3
  15. 2025Imagine to Hear: Auditory Knowledge Generation can be an Effective Assistant for Language Modelscited by 2
  16. 2025DreamAudio: Customized Text-to-Audio Generation with Diffusion Modelscited by 2
  17. 2025Does Hearing Help Seeing? Investigating Audio-Video Joint Denoising for Video Generationcited by 2
  18. 2026Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decodingcited by 2
  19. 2026Audio ControlNet for Fine-Grained Audio Generation and Editingcited by 2
  20. 2026Few-shot Acoustic Synthesis with Multimodal Flow Matchingcited by 2
  21. 2026FoleyDirector: Fine-Grained Temporal Steering for Video-to-Audio Generation via Structured Scriptscited by 2
  22. 2025DegDiT: Controllable Audio Generation with Dynamic Event Graph Guided Diffusion Transformercited by 1
  23. 2025MotionFlux: Efficient Text-Guided Motion Generation through Rectified Flow Matching and Preference Alignmentcited by 1
  24. 2025FlowBind: Efficient Any-to-Any Generation with Bidirectional Flowscited by 1
  25. 2026Membership Inference Attacks against Large Audio Language Modelscited by 1
  26. 2026UNISON: A Unified Sound Generation and Editing Framework via Deep LLM Fusioncited by 1
  27. 2026AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generationcited by 1
  28. 2026FreeSonic: Training-Free Temporal-Aware Decoupled Attention for Precise Audio Editingcited by 1
  29. 2026MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generationcited by 1
  30. 2026OTCache: Optimal Transport for Geometry-Aware Caching in Diffusion Modelscited by 1
  31. 2026FlowSep 2: Self-Supervised Flow Matching for Language-Queried Audio Source Separationcited by 1
  32. 2025Sat2Sound: A Unified Framework for Zero-Shot Soundscape Mappingcited by 0
  33. 2025TTMBA: Towards Text To Multiple Sources Binaural Audio Generationcited by 0
  34. 2025MultiSoundGen: Video-to-Audio Generation for Multi-Event Scenarios via SlowFast Contrastive Audio-Visual Pretraining and Direct Preference Optimizationcited by 0
  35. 2025ETC: training-free diffusion models acceleration with Error-aware Trend Consistencycited by 0
  36. 2026Investigating Group Relative Policy Optimization for Diffusion Transformer based Text-to-Audio Generationcited by 0
  37. 2026ImmersiveTTS: Environment-Aware Text-to-Speech with Multimodal Diffusion Transformer and Domain-Specific Representation Alignmentcited by 0
  38. 2026RFM-Editing 2: Text-Guided Audio Editing with Rectified Flow Matching and Coarse-to-Fine Diffusion Transformerscited by 0

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Resonate: Reinforcing Text-to-Audio Generation via Online Feedback from Large Audio Language Models ↗
  2. 2026TinyAudio: Compact and Efficient Text-to-Audio Generation for Low-Resource Deployment ↗
  3. 2023ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation
  4. 2025UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalities
  5. 2023Retrieval-Augmented Text-to-Audio Generation
  6. 2024AudioComposer: Towards Fine-grained Audio Generation with Natural Language Descriptions
  7. 2024Audiobox TTA-RAG: Improving Zero-Shot and Few-Shot Text-To-Audio with Retrieval-Augmented Generation
  8. 2024ETTA: Elucidating the Design Space of Text-to-Audio Models