mlsanityData to 7 Oct 2026

LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation

arXiv 2406.08203 ↗First posted 12 Jun 2024eess.AS cs.SDcited by 24

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Recently, the application of diffusion models has facilitated the significant development of speech and audio generation. Nevertheless, the quality of samples generated by diffusion models still needs improvement. And the effectiveness of the method is accompanied by the extensive number of sampling steps, leading to an extended synthesis time necessary for generating high-quality audio. Previous Text-to-Audio (TTA) methods mostly used diffusion models in the latent space for audio generation. In this paper, we explore the integration of the Flow Matching (FM) model into the audio latent space for audio generation. The FM is an alternative simulation-free method that trains continuous normalization flows (CNF) based on regressing vector fields. We demonstrate that our model significantly enhances the quality of generated audio samples, achieving better performance than prior models. Moreover, it reduces the number of inference steps to ten steps almost without sacrificing performance.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 25 of its 31 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

25 of 31 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation2 Jan
  2. 2023Audiobox: Unified Audio Generation with Natural Language Prompts25 Dec
  3. 2023Generative Pre-training for Speech with Flow Matching25 Oct
  4. 2023ReFlow-TTS: A Rectified Flow Model for High-fidelity Text-to-Speech29 Sep
  5. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining10 Aug
  6. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model11 May
  7. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Models29 Jan
  8. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generation20 Jul
  9. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech13 Jul

Other references · 16

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2018Neural Ordinary Differential Equationscited by 3,733
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  10. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  11. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  12. 2022AudioGen: Textually Guided Audio Generationcited by 292
4 more
  1. 2022Masked Autoencoders that Listencited by 271
  2. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  3. 2023Guided Flows for Generative Modeling and Decision Makingcited by 99
  4. 2023MM-TTS: Multi-modal Prompt based Style Transfer for Expressive Text-to-Speech Synthesiscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 12; 2026: 9.

24 papers.

Sort:

By area:

  1. 2024Taming Data and Transformers for Audio Generationcited by 28
  2. 2025SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modelingcited by 26
  3. 2025MeanAudio: Fast and Faithful Text-to-Audio Generation with Mean Flowscited by 14
  4. 2026Faster Inference of Flow-Based Generative Models via Improved Data-Noise Couplingcited by 6
  5. 2024Local Flow Matching Generative Modelscited by 5
  6. 2024YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controlscited by 5
  7. 2025Discl-VC: Disentangled Discrete Tokens and In-Context Learning for Controllable Zero-Shot Voice Conversioncited by 5
  8. 2025UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalitiescited by 5
  9. 2025Is There a Better Source Distribution than Gaussian? Exploring Source Distributions for Image Flow Matchingcited by 4
  10. 2025SlimSpeech: Lightweight and Efficient Text-to-Speech with Slim Rectified Flowcited by 3
  11. 2025Generative Unordered Flow for Set-Structured Data Generationcited by 2
  12. 2025AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusioncited by 2
12 more
  1. 2025DreamAudio: Customized Text-to-Audio Generation with Diffusion Modelscited by 2
  2. 2025Towards Stable and Structured Time Series Generation with Perturbation-Aware Flow Matchingcited by 2
  3. 2026Preconditioned Flow Matchingcited by 2
  4. 2026Geometry-Aware Image Flow Matchingcited by 2
  5. 2025UniVoice: Unifying Autoregressive ASR and Flow-Matching based TTS with Large Language Modelscited by 1
  6. 2025GuitarFlow: Realistic Electric Guitar Synthesis From Tablatures via Flow Matching and Style Transfercited by 1
  7. 2026AG-REPA: Causal Layer Selection for Representation Alignment in Audio Flow Matchingcited by 1
  8. 2026UAT: Unified Audio-Text Diffusion for Audio Generation, Editing, and Captioningcited by 1
  9. 2026Investigating Group Relative Policy Optimization for Diffusion Transformer based Text-to-Audio Generationcited by 0
  10. 2026HoliDubber: Holistic Video Dubbing for Complex Acoustic Scenes via Text-Guided Audio Synthesiscited by 0
  11. 2026A Theory on Flow Matching with Neural Networkscited by 0
  12. 2026RIPPLE: Generating Multi-Channel Phase, Not Recovering Itcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation
  2. 2024AudioLCM: Text-to-Audio Generation with Latent Consistency Models
  3. 2023Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation
  4. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching
  5. 2025Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementation
  6. 2026UNITE-AUDIO: Joint Learning of Continuous Tokenization and Latent Flow Matching for Text-to-Audio Generation
  7. 2024FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation
  8. 2024PTQ4ADM: Post-Training Quantization for Efficient Text Conditional Audio Diffusion Models