mlsanityData to 14 Oct 2026

A Survey on World Models Grounded in Acoustic Physical Information

arXiv 2506.13833 ↗First posted 16 Jun 2025cs.SD cs.AI cs.ROcited by 1

On the map: World models → Latent predictive world models

Abstract (arXiv)

This survey provides a comprehensive overview of the emerging field of world models grounded in the foundation of acoustic physical information. It examines the theoretical underpinnings, essential methodological frameworks, and recent technological advancements in leveraging acoustic signals for high-fidelity environmental perception, causal physical reasoning, and predictive simulation of dynamic events. The survey explains how acoustic signals, as direct carriers of mechanical wave energy from physical events, encode rich, latent information about material properties, internal geometric structures, and complex interaction dynamics. Specifically, this survey establishes the theoretical foundation by explaining how fundamental physical laws govern the encoding of physical information within acoustic signals. It then reviews the core methodological pillars, including Physics-Informed Neural Networks (PINNs), generative models, and self-supervised multimodal learning frameworks. Furthermore, the survey details the significant applications of acoustic world models in robotics, autonomous driving, healthcare, and finance. Finally, it systematically outlines the important technical and ethical challenges while proposing a concrete roadmap for future research directions toward robust, causal, uncertainty-aware, and responsible acoustic intelligence. These elements collectively point to a research pathway towards embodied active acoustic intelligence, empowering AI systems to construct an internal "intuitive physics" engine through sound.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 33 of its 88 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

33 of 88 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  7. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  8. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  9. 2018World Modelscited by 1,513
  10. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  11. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
21 more
  1. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  2. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  3. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
  4. 2017Look, Listen and Learncited by 458
  5. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 342
  6. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  7. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
  8. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 250
  9. 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 216
  10. 2018Learning to Localize Sound Source in Visual Scenescited by 189
  11. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 156
  12. 2019Co-Separating Sounds of Visual Objectscited by 110
  13. 2022HEAR: Holistic Evaluation of Audio Representationscited by 78
  14. 2020Equivariant Neural Renderingcited by 48
  15. 2022MUGEN: A Playground for Video-Audio-Text Multimodal Understanding and GENerationcited by 20
  16. 2021Audio-visual Representation Learning for Anomaly Events Detection in Crowdscited by 13
  17. 2019Robot Sound Interpretation: Combining Sight and Sound in Learning-Based Controlcited by 7
  18. 2024A decade of DCASE: Achievements, practices, evaluations and future challengescited by 7
  19. 2024Physics-Informed Neural Network for Volumetric Sound field Reconstruction of Speech Signalscited by 6
  20. 2018Deep Room Recognition Using Inaudible Echoscited by 2
  21. 2025A Synergistic Framework of Nonlinear Acoustic Computing and Reinforcement Learning for Real-World Human-Robot Interactioncited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026BinauralVAE: Spatial Audio Reconstruction For World Models
  2. 2026FillGauss: Fine-Grained Filling-Aware Impact Sound Generation for 3D Gaussian Splatting
  3. 2025Audio-Visual World Models: Learning Physically Grounded Multisensory Dynamics
  4. 2026PhysFieldBench: Can Multimodal Models Understand Physical Fields?
  5. 2023Physics-Driven Diffusion Models for Impact Sound Synthesis from Videos
  6. 2024Urban Sound Propagation: a Benchmark for 1-Step Generative Modeling of Complex Physical Systems
  7. 2025Embodied Tactile Perception of Soft Objects Properties ↗
  8. 2026GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models