Few-shot Acoustic Synthesis with Multimodal Flow Matching
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
Generating audio that is acoustically consistent with a scene is essential for immersive virtual environments. Recent neural acoustic field methods enable spatially continuous sound rendering but remain scene-specific, requiring dense audio measurements and costly training for each environment. Few-shot approaches improve scalability across rooms but still rely on multiple recordings and, being deterministic, fail to capture the inherent uncertainty of scene acoustics under sparse context. We introduce flow-matching acoustic generation (FLAC), a probabilistic method for few-shot acoustic synthesis that models the distribution of plausible room impulse responses (RIRs) given minimal scene context. FLAC leverages a diffusion transformer trained with a flow-matching objective to generate RIRs at arbitrary positions in novel scenes, conditioned on spatial, geometric, and acoustic cues. FLAC outperforms state-of-the-art eight-shot baselines with one-shot on both the AcousticRooms and Hearing Anything Anywhere datasets. To complement standard perceptual metrics, we further introduce AGREE, a joint acoustic-geometry embedding, enabling geometry-consistent evaluation of generated RIRs through retrieval and distributional metrics. This work is the first to apply generative flow matching to explicit RIR synthesis, establishing a new direction for robust and data-efficient acoustic synthesis.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 75 of its 89 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
75 of 89 references.
Closest ancestors · 9
Same direction, newest first
- 2025BinauralFlow: A Causal and Streamable Approach for High-Quality Binaural Speech Synthesis with Flow Matching Models28 May
- 2024TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization30 Dec
- 2024Long-form music generation with latent diffusion16 Apr
- 2024Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization15 Apr
- 2023Controllable Music Production with Diffusion Models and Guidance Gradients1 Nov
- 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching10 Sep
- 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining10 Aug
- 2023Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation29 May
- 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model24 Apr
Other references · 66
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
38 more
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2025DINOv3cited by 1,297
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
- 2018Open3D: A Modern Library for 3D Data Processingcited by 717
- 2022High Fidelity Neural Audio Compressioncited by 673
- 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
- 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
- 2022Rectified Flow: A Marginal Preserving Approach to Optimal Transportcited by 277
- 2020Audio-Visual Instance Discrimination with Cross-Modal Agreementcited by 192
- 2024Stable Audio Opencited by 192
- 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 97
- 2019Look, Listen, and Act: Towards Audio-Visual Embodied Navigationcited by 95
- 2018Self-Supervised Generation of Spatial Audio for 360 Videocited by 93
- 2022SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learningcited by 93
- 2022Learning Neural Acoustic Fieldscited by 65
- 2024EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberationcited by 62
- 2020Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separationcited by 55
- 2020VisualEchoes: Spatial Image Representation Learning through Echolocationcited by 54
- 2023AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesiscited by 49
- 2022Few-Shot Audio-Visual Learning of Environment Acousticscited by 48
- 2022Visual Acoustic Matchingcited by 45
- 2021Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Soundscited by 40
- 2021FAST-RIR: Fast neural diffuse room impulse response generatorcited by 35
- 2021Image2Reverb: Cross-Modal Reverb Impulse Response Synthesiscited by 34
- 2020Audio-Visual Floorplan Reconstructioncited by 32
- 2019BatVision: Learning to See 3D Spatial Layout with Two Earscited by 26
- 2020Semantic Object Prediction and Spatial Sound Super-Resolution with Binaural Soundscited by 26
- 2021Learning Audio-Visual Dereverberationcited by 26
- 2022GWA: A Large High-Quality Acoustic Dataset for Audio Processingcited by 25
- 2021Beyond Image to Depth: Improving Depth Prediction using Echoescited by 24
- 2023Neural Acoustic Context Field: Rendering Realistic Room Impulse Response With Neural Fieldscited by 23
- 2023AV-RIR: Audio-Visual Room Impulse Response Estimationcited by 20
- 2024Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmarkcited by 20
50 most cited of 66 All 66 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025PromptReverb: Multimodal Room Impulse Response Generation Through Latent Rectified Flow Matching
- 2024Diff-SAGe: End-to-End Spatial Audio Generation Using Diffusion Models
- 2025Room Impulse Response Generation Conditioned on Acoustic Parameters
- 2026Audible World Models: Spatially Aware Sound Generation for 3D Worlds
- 2026Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer
- 2026Flow-HOA: Generative Joint Optimization for Ambisonics Encoding via Flow Matching
- 2026Scene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worlds
- 2025PAVAS: Physics-Aware Video-to-Audio Synthesis