mlsanityData to 7 Oct 2026

PhysWave: Physics-Guided Latent Diffusion Models for Controllable Spatial Audio Generation

arXiv 2608.29549 ↗First posted 30 Aug 2026cs.SD cs.AI cs.MMcited by 0

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Text-to-spatial audio generation, such as text-to-First-Order Ambisonics (FOA), provides a convenient way to create spatial audio for billion-dollar gaming and film industries. However, existing text-to-FOA methods are largely data-driven and may produce audio that violates acoustic relations between source direction and distance. They also separate descriptive and parametric control, forcing users to trade usability for precision. In this paper, we present PhysWave, a physics-guided latent diffusion model for controllable text-to-FOA generation. PhysWave unifies natural-language and trajectory control through a shared waypoint-caption representation, and augments diffusion training with two differentiable acoustic priors: spherical-harmonic direction consistency and inverse-square distance consistency. To support dynamic spatial generation, we further construct a 300K-clip FOA dataset with diverse sound categories and source trajectories. Extensive results show that the proposed priors help PhysWave generate spatially consistent FOA audio while maintaining competitive audio quality. Further analyses show that these physics priors improve spatial consistency during training and can also be used as inference-time guidance for training-free spatial refinement.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 21 of its 43 references that are in the map. First posted on arXiv 30 Aug 2026.

Builds on

21 of 43 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025DualSpec: Text-to-spatial-audio Generation via Dual-Spectrogram Guided Diffusion Model26 Feb
  2. 2024Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation14 Oct
  3. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining10 Aug
  4. 2023Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation29 May
  5. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models30 Jan

Other references · 16

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  3. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 563
  4. 2019Clotho: An Audio Captioning Datasetcited by 354
  5. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 266
  6. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 178
  7. 2022A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstructioncited by 150
  8. 2023STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Eventscited by 39
  9. 2017Direction of arrival estimation for multiple sound sources using convolutional recurrent neural networkcited by 27
  10. 2024PID: Physics-Informed Diffusion Model for Infrared Image Generationcited by 17
  11. 2025ViSAGe: Video-to-Spatial Audio Generationcited by 11
  12. 2024Physics-Informed Neural Network for Volumetric Sound field Reconstruction of Speech Signalscited by 6
4 more
  1. 2025AudioSpa: Spatializing Sound Events with Textcited by 3
  2. 2025FOA Tokenizer: Low-bitrate Neural Codec for First Order Ambisonics with Spatial Consistency Losscited by 2
  3. 2025In-the-wild Audio Spatialization with Flexible Text-guided Localizationcited by 1
  4. 2025A PDE-Informed Latent Diffusion Model for 2-m Temperature Downscalingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Generating Moving 3D Soundscapes with Latent Diffusion Models
  2. 2024ImmerseDiffusion: A Generative Spatial Audio Latent Diffusion Model
  3. 2024Diff-SAGe: End-to-End Spatial Audio Generation Using Diffusion Models
  4. 2025Audio Palette: A Diffusion Transformer with Multi-Signal Conditioning for Controllable Foley Synthesis
  5. 2025ControlAudio: Tackling Text-Guided, Timing-Indicated and Intelligible Audio Generation via Progressive Diffusion Modeling
  6. 2026Low-Resource Guidance for Controllable Latent Audio Diffusion
  7. 2026Diffusion Models for Joint Audio-Video Generation
  8. 2025PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation