mlsanityData to 14 Oct 2026

Do Joint Audio-Video Generation Models Understand Physics?

arXiv 2605.07061 ↗First posted 8 May 2026cs.SD cs.AI cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Joint audio-video generation models are rapidly approaching professional production quality, raising a central question: do they understand audio-visual physics, or merely generate plausible sounds and frames that violate real-world consistency? We introduce AV-Phys Bench, a benchmark for evaluating physical commonsense in joint audio-video generation. AV-Phys Bench tests models across three scene categories: Steady State, Event Transition, and Environment Transition. It covers physics-grounded subcategories drawn from real-world scenes, plus Anti-AV-Physics prompts that deliberately request physically inconsistent audio-video behavior. Each generation is evaluated along five dimensions: visual semantic adherence, audio semantic adherence, visual physical commonsense, audio physical commonsense, and cross-modal physical commonsense. Across three proprietary and four open-source models, we find that Seedance 2.0 performs best overall, but all models remain far from robust physical understanding. Performance drops sharply on event-driven and environment-driven transitions, and even strong proprietary systems collapse on Anti-AV-Physics prompts. We further introduce AV-Phys Agent, a ReAct-style evaluator that combines a multimodal language model with deterministic acoustic measurement tools, producing rankings that closely align with human ratings. Our results identify cross-modal physical consistency and transition-driven scene dynamics as key open challenges for joint audio-video generation.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 41 of its 47 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

41 of 47 references.

References · 41

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  4. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  5. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  6. 2025OpenAI GPT-5 System Cardcited by 989
  7. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  8. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  9. 2024Genie: Generative Interactive Environmentscited by 747
  10. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
  11. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  12. 2023A Survey on Video Diffusion Modelscited by 206
29 more
  1. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  2. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
  3. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 174
  4. 2026Qwen3.5-Omni Technical Reportcited by 174
  5. 2025Do generative video models understand physical principles?cited by 161
  6. 2026Seedance 2.0: Advancing Video Generation for World Complexitycited by 160
  7. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 149
  8. 2019Look, Listen, and Act: Towards Audio-Visual Embodied Navigationcited by 96
  9. 2022SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learningcited by 95
  10. 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 88
  11. 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 83
  12. 2023Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptationcited by 69
  13. 2025UniVerse-1: Unified Audio-Video Generation via Stitching of Expertscited by 61
  14. 2025T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generationcited by 39
  15. 2024TAVGBench: Benchmarking Text to Audible-Video Generationcited by 34
  16. 2024Tell What You Hear From What You See -- Video to Audio Generation Through Textcited by 34
  17. 2025VABench: A Comprehensive Benchmark for Audio-Video Generationcited by 30
  18. 2026JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generationcited by 25
  19. 2026Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Modelcited by 25
  20. 2025T2AV-Compass: Towards Unified Evaluation for Text-to-Audio-Video Generationcited by 24
  21. 2024From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generationcited by 23
  22. 2025SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearingcited by 15
  23. 2023CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environmentscited by 14
  24. 2025PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generationcited by 11
  25. 2024SAVGBench: Benchmarking Spatially Aligned Audio-Video Generationcited by 10
  26. 2026A Systematic Post-Train Framework for Video Generationcited by 5
  27. 2024Virbo: Multimodal Multilingual Avatar Video Generation in Digital Marketingcited by 4
  28. 2026SonicBench: Dissecting the Physical Perception Bottleneck in Large Audio Language Modelscited by 2
  29. 2026OmniSonic: Towards Universal and Holistic Audio Generation from Video and Textcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Models
  2. 2025PAVAS: Physics-Aware Video-to-Audio Synthesis
  3. 2025PhysVidBench: Language-Grounded Evaluation of Physical Commonsense in Text-to-Video Models
  4. 2025Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model
  5. 2026AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation
  6. 2025Bootstrapping Physics-Grounded Video Generation through VLM-Guided Iterative Self-Refinement
  7. 2026AVBench: Human-Aligned and Automated Evaluation Benchmark for Audio-Video Generative Models
  8. 2026Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioning