A Survey on World Models Grounded in Acoustic Physical Information
On the map: World models → Latent predictive world models
Abstract (arXiv)
This survey provides a comprehensive overview of the emerging field of world models grounded in the foundation of acoustic physical information. It examines the theoretical underpinnings, essential methodological frameworks, and recent technological advancements in leveraging acoustic signals for high-fidelity environmental perception, causal physical reasoning, and predictive simulation of dynamic events. The survey explains how acoustic signals, as direct carriers of mechanical wave energy from physical events, encode rich, latent information about material properties, internal geometric structures, and complex interaction dynamics. Specifically, this survey establishes the theoretical foundation by explaining how fundamental physical laws govern the encoding of physical information within acoustic signals. It then reviews the core methodological pillars, including Physics-Informed Neural Networks (PINNs), generative models, and self-supervised multimodal learning frameworks. Furthermore, the survey details the significant applications of acoustic world models in robotics, autonomous driving, healthcare, and finance. Finally, it systematically outlines the important technical and ethical challenges while proposing a concrete roadmap for future research directions toward robust, causal, uncertainty-aware, and responsible acoustic intelligence. These elements collectively point to a research pathway towards embodied active acoustic intelligence, empowering AI systems to construct an internal "intuitive physics" engine through sound.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 33 of its 88 references that are in the map. First posted on arXiv 16 Jun 2025.
Builds on
33 of 88 references.
References · 33
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2018World Modelscited by 1,513
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
- 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
21 more
- 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
- 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
- 2017Look, Listen and Learncited by 458
- 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 342
- 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
- 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
- 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 250
- 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 216
- 2018Learning to Localize Sound Source in Visual Scenescited by 189
- 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 156
- 2019Co-Separating Sounds of Visual Objectscited by 110
- 2022HEAR: Holistic Evaluation of Audio Representationscited by 78
- 2020Equivariant Neural Renderingcited by 48
- 2022MUGEN: A Playground for Video-Audio-Text Multimodal Understanding and GENerationcited by 20
- 2021Audio-visual Representation Learning for Anomaly Events Detection in Crowdscited by 13
- 2019Robot Sound Interpretation: Combining Sight and Sound in Learning-Based Controlcited by 7
- 2024A decade of DCASE: Achievements, practices, evaluations and future challengescited by 7
- 2024Physics-Informed Neural Network for Volumetric Sound field Reconstruction of Speech Signalscited by 6
- 2018Deep Room Recognition Using Inaudible Echoscited by 2
- 2025A Synergistic Framework of Nonlinear Acoustic Computing and Reinforcement Learning for Real-World Human-Robot Interactioncited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026BinauralVAE: Spatial Audio Reconstruction For World Models
- 2026FillGauss: Fine-Grained Filling-Aware Impact Sound Generation for 3D Gaussian Splatting
- 2025Audio-Visual World Models: Learning Physically Grounded Multisensory Dynamics
- 2026PhysFieldBench: Can Multimodal Models Understand Physical Fields?
- 2023Physics-Driven Diffusion Models for Impact Sound Synthesis from Videos
- 2024Urban Sound Propagation: a Benchmark for 1-Step Generative Modeling of Complex Physical Systems
- 2025Embodied Tactile Perception of Soft Objects Properties ↗
- 2026GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models