# PhysWave: Physics-Guided Latent Diffusion Models for Controllable Spatial Audio Generation

arXiv: https://arxiv.org/abs/2608.29549 · HTML: https://mlsanity.com/paper/2608.29549/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 30 Aug 2026.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 21 of 21)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Clotho: An Audio Captioning Dataset](https://mlsanity.com/paper/1910.09387/) (2019)
- [AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining](https://mlsanity.com/paper/2308.05734/) (2023)
- [Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models](https://mlsanity.com/paper/2301.12661/) (2023)
- [WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research](https://mlsanity.com/paper/2303.17395/) (2023)
- [Fast Timing-Conditioned Latent Audio Diffusion](https://mlsanity.com/paper/2402.04825/) (2024)
- [A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction](https://mlsanity.com/paper/2211.14680/) (2022)
- [Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation](https://mlsanity.com/paper/2305.18474/) (2023)
- [STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events](https://mlsanity.com/paper/2306.09126/) (2023)
- [Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network](https://mlsanity.com/paper/1710.10059/) (2017)
- [Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation](https://mlsanity.com/paper/2410.10676/) (2024)
- [PID: Physics-Informed Diffusion Model for Infrared Image Generation](https://mlsanity.com/paper/2407.09299/) (2024)
- [ViSAGe: Video-to-Spatial Audio Generation](https://mlsanity.com/paper/2506.12199/) (2025)
- [Physics-Informed Neural Network for Volumetric Sound field Reconstruction of Speech Signals](https://mlsanity.com/paper/2403.09524/) (2024)
- [AudioSpa: Spatializing Sound Events with Text](https://mlsanity.com/paper/2502.11219/) (2025)
- [DualSpec: Text-to-spatial-audio Generation via Dual-Spectrogram Guided Diffusion Model](https://mlsanity.com/paper/2502.18952/) (2025)
- [FOA Tokenizer: Low-bitrate Neural Codec for First Order Ambisonics with Spatial Consistency Loss](https://mlsanity.com/paper/2510.22241/) (2025)
- [In-the-wild Audio Spatialization with Flexible Text-guided Localization](https://mlsanity.com/paper/2506.00927/) (2025)
- [A PDE-Informed Latent Diffusion Model for 2-m Temperature Downscaling](https://mlsanity.com/paper/2510.23866/) (2025)

## Abstract (arXiv)

Text-to-spatial audio generation, such as text-to-First-Order Ambisonics (FOA), provides a convenient way to create spatial audio for billion-dollar gaming and film industries. However, existing text-to-FOA methods are largely data-driven and may produce audio that violates acoustic relations between source direction and distance. They also separate descriptive and parametric control, forcing users to trade usability for precision. In this paper, we present PhysWave, a physics-guided latent diffusion model for controllable text-to-FOA generation. PhysWave unifies natural-language and trajectory control through a shared waypoint-caption representation, and augments diffusion training with two differentiable acoustic priors: spherical-harmonic direction consistency and inverse-square distance consistency. To support dynamic spatial generation, we further construct a 300K-clip FOA dataset with diverse sound categories and source trajectories. Extensive results show that the proposed priors help PhysWave generate spatially consistent FOA audio while maintaining competitive audio quality. Further analyses show that these physics priors improve spatial consistency during training and can also be used as inference-time guidance for training-free spatial refinement.