mlsanityData to 7 Oct 2026

Few-shot Acoustic Synthesis with Multimodal Flow Matching

arXiv 2603.19176 ↗First posted 19 Mar 2026cs.SD cs.CV eess.AScited by 2

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Generating audio that is acoustically consistent with a scene is essential for immersive virtual environments. Recent neural acoustic field methods enable spatially continuous sound rendering but remain scene-specific, requiring dense audio measurements and costly training for each environment. Few-shot approaches improve scalability across rooms but still rely on multiple recordings and, being deterministic, fail to capture the inherent uncertainty of scene acoustics under sparse context. We introduce flow-matching acoustic generation (FLAC), a probabilistic method for few-shot acoustic synthesis that models the distribution of plausible room impulse responses (RIRs) given minimal scene context. FLAC leverages a diffusion transformer trained with a flow-matching objective to generate RIRs at arbitrary positions in novel scenes, conditioned on spatial, geometric, and acoustic cues. FLAC outperforms state-of-the-art eight-shot baselines with one-shot on both the AcousticRooms and Hearing Anything Anywhere datasets. To complement standard perceptual metrics, we further introduce AGREE, a joint acoustic-geometry embedding, enabling geometry-consistent evaluation of generated RIRs through retrieval and distributional metrics. This work is the first to apply generative flow matching to explicit RIR synthesis, establishing a new direction for robust and data-efficient acoustic synthesis.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 75 of its 89 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

75 of 89 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025BinauralFlow: A Causal and Streamable Approach for High-Quality Binaural Speech Synthesis with Flow Matching Models28 May
  2. 2024TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization30 Dec
  3. 2024Long-form music generation with latent diffusion16 Apr
  4. 2024Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization15 Apr
  5. 2023Controllable Music Production with Diffusion Models and Guidance Gradients1 Nov
  6. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching10 Sep
  7. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining10 Aug
  8. 2023Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation29 May
  9. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model24 Apr

Other references · 66

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  9. 2022Scalable Diffusion Models with Transformerscited by 5,931
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2022Flow Matching for Generative Modelingcited by 5,426
  12. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
38 more
  1. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  2. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  3. 2025DINOv3cited by 1,297
  4. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  5. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  6. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  7. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
  8. 2022High Fidelity Neural Audio Compressioncited by 673
  9. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  10. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  11. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  12. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  13. 2022Rectified Flow: A Marginal Preserving Approach to Optimal Transportcited by 277
  14. 2020Audio-Visual Instance Discrimination with Cross-Modal Agreementcited by 192
  15. 2024Stable Audio Opencited by 192
  16. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 97
  17. 2019Look, Listen, and Act: Towards Audio-Visual Embodied Navigationcited by 95
  18. 2018Self-Supervised Generation of Spatial Audio for 360 Videocited by 93
  19. 2022SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learningcited by 93
  20. 2022Learning Neural Acoustic Fieldscited by 65
  21. 2024EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberationcited by 62
  22. 2020Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separationcited by 55
  23. 2020VisualEchoes: Spatial Image Representation Learning through Echolocationcited by 54
  24. 2023AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesiscited by 49
  25. 2022Few-Shot Audio-Visual Learning of Environment Acousticscited by 48
  26. 2022Visual Acoustic Matchingcited by 45
  27. 2021Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Soundscited by 40
  28. 2021FAST-RIR: Fast neural diffuse room impulse response generatorcited by 35
  29. 2021Image2Reverb: Cross-Modal Reverb Impulse Response Synthesiscited by 34
  30. 2020Audio-Visual Floorplan Reconstructioncited by 32
  31. 2019BatVision: Learning to See 3D Spatial Layout with Two Earscited by 26
  32. 2020Semantic Object Prediction and Spatial Sound Super-Resolution with Binaural Soundscited by 26
  33. 2021Learning Audio-Visual Dereverberationcited by 26
  34. 2022GWA: A Large High-Quality Acoustic Dataset for Audio Processingcited by 25
  35. 2021Beyond Image to Depth: Improving Depth Prediction using Echoescited by 24
  36. 2023Neural Acoustic Context Field: Rendering Realistic Room Impulse Response With Neural Fieldscited by 23
  37. 2023AV-RIR: Audio-Visual Room Impulse Response Estimationcited by 20
  38. 2024Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmarkcited by 20

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025PromptReverb: Multimodal Room Impulse Response Generation Through Latent Rectified Flow Matching
  2. 2024Diff-SAGe: End-to-End Spatial Audio Generation Using Diffusion Models
  3. 2025Room Impulse Response Generation Conditioned on Acoustic Parameters
  4. 2026Audible World Models: Spatially Aware Sound Generation for 3D Worlds
  5. 2026Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer
  6. 2026Flow-HOA: Generative Joint Optimization for Ambisonics Encoding via Flow Matching
  7. 2026Scene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worlds
  8. 2025PAVAS: Physics-Aware Video-to-Audio Synthesis