mlsanityData to 7 Oct 2026

WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generation

arXiv 2503.08153 ↗First posted 11 Mar 2025cs.CVcited by 87

Not on the map. Why?

Abstract (arXiv)

Recent rapid advancements in text-to-video (T2V) generation, such as SoRA and Kling, have shown great potential for building world simulators. However, current T2V models struggle to grasp abstract physical principles and generate videos that adhere to physical laws. This challenge arises primarily from a lack of clear guidance on physical information due to a significant gap between abstract physical principles and generation models. To this end, we introduce the World Simulator Assistant (WISA), an effective framework for decomposing and incorporating physical principles into T2V models. Specifically, WISA decomposes physical principles into textual physical descriptions, qualitative physical categories, and quantitative physical properties. To effectively embed these physical attributes into the generation process, WISA incorporates several key designs, including Mixture-of-Physical-Experts Attention (MoPA) and a Physical Classifier, enhancing the model's physics awareness. Furthermore, most existing datasets feature videos where physical phenomena are either weakly represented or entangled with multiple co-occurring processes, limiting their suitability as dedicated resources for learning explicit physical principles. We propose a novel video dataset, WISA-32K, collected based on qualitative physical categories. It consists of 32,000 videos, representing 17 physical laws across three domains of physics: dynamics, thermodynamics, and optics. Experimental results demonstrate that WISA can effectively enhance the compatibility of T2V models with real-world physical laws, achieving a considerable improvement on the VideoPhy benchmark. The visual exhibitions of WISA and WISA-32K are available in the https://360cvgroup.github.io/WISA/.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Builds on 34 of its 45 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

34 of 45 references.

References · 34

  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  7. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  8. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  10. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  11. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  12. 2023ModelScope Text-to-Video Technical Reportcited by 651
22 more
  1. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  2. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  3. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
  4. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  5. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
  6. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  7. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
  8. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  9. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
  10. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 167
  11. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 163
  12. 2025Do generative video models understand physical principles?cited by 151
  13. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  14. 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 132
  15. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  16. 2024Pandora: Towards General World Model with Natural Language Actions and Video Statescited by 83
  17. 2024PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generationcited by 77
  18. 2024MotionCraft: Physics-based Zero-Shot Video Generationcited by 46
  19. 2022Differentiable Physics Simulation of Dynamics-Augmented Neural Objectscited by 45
  20. 2024MoH: Multi-Head Attention as Mixture-of-Head Attentioncited by 41
  21. 2024HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generationcited by 39
  22. 2024FancyVideo: Towards Dynamic and Consistent Video Generation via Cross-frame Textual Guidancecited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 30; 2026: 57.

87 papers.

Sort:

By area:

  1. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 88
  2. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 40
  3. 2025T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generationcited by 35
  4. 2025"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Modelscited by 34
  5. 2026Inference-time Physics Alignment of Video Generative Models with Latent World Modelscited by 32
  6. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  7. 2025PAI-Bench: A Comprehensive Benchmark For Physical AIcited by 30
  8. 2026Rethinking Video Generation Model for the Embodied Worldcited by 30
  9. 2025What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewardscited by 23
  10. 2025Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generationcited by 20
  11. 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Modelscited by 19
  12. 2025PlanGen: Towards Unified Layout Planning and Image Generation in Auto-Regressive Vision Language Modelscited by 18
38 more
  1. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  2. 2026Self-Refining Video Samplingcited by 14
  3. 2025Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluationcited by 13
  4. 2025Enhancing Physical Plausibility in Video Generation by Reasoning the Implausibilitycited by 13
  5. 2025MoAlign: Motion-Centric Representation Alignment for Video Diffusion Modelscited by 13
  6. 2026Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoningcited by 13
  7. 2025RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligencecited by 12
  8. 2026SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Modelscited by 12
  9. 2025ProPhy: Progressive Physical Alignment for Dynamic World Simulationcited by 11
  10. 2025PhysMaster: Mastering Physical Representation for Video Generation via Reinforcement Learningcited by 10
  11. 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Modelscited by 10
  12. 2026Phantom: Physics-Infused Video Generation via Joint Modeling of Visual and Latent Physical Dynamicscited by 10
  13. 2026NEWTON: Agentic Planning for Physically Grounded Video Generationcited by 10
  14. 2026PhysInOne: Visual Physics Learning and Reasoning in One Suitecited by 9
  15. 2026ACWM-Phys: Investigating Generalized Physical Interaction in Action-Conditioned Video World Modelscited by 9
  16. 2025RAGAR: Retrieval Augmented Personalized Image Generation Guided by Recommendationcited by 8
  17. 2025RDPO: Real Data Preference Optimization for Physics Consistency Video Generationcited by 8
  18. 2025PhyBlock: A Progressive Benchmark for Physical Understanding and Planning via 3D Block Assemblycited by 7
  19. 2025UnityVideo: Unified Multi-Modal Multi-Task Learning for Enhancing World-Aware Video Generationcited by 7
  20. 2026OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generationcited by 7
  21. 2026VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine Systemcited by 7
  22. 2025Lay2Story: Extending Diffusion Transformers for Layout-Togglable Story Generationcited by 6
  23. 2025MoFu: Scale-Aware Modulation and Fourier Fusion for Multi-Subject Video Generationcited by 6
  24. 2026Chain of Event-Centric Causal Thought for Physically Plausible Video Generationcited by 6
  25. 2026Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generationcited by 6
  26. 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 6
  27. 2025MoReGen: Multi-Agent Motion-Reasoning Engine for Code-based Text-to-Video Synthesiscited by 5
  28. 2026InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generationcited by 5
  29. 2026PhysVid: Physics Aware Local Conditioning for Generative Video Modelscited by 5
  30. 2025Physics-Grounded Motion Forecasting via Equation Discovery for Trajectory-Guided Image-to-Video Generationcited by 4
  31. 2025PhyDetEx: Detecting and Explaining the Physical Plausibility of T2V Modelscited by 4
  32. 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Modelscited by 4
  33. 2026From Ideal to Real: Stable Video Object Removal under Imperfect Conditionscited by 4
  34. 2026LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generationcited by 4
  35. 2026VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimizationcited by 4
  36. 2026TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RLcited by 4
  37. 2025Bootstrapping Physics-Grounded Video Generation through VLM-Guided Iterative Self-Refinementcited by 3
  38. 2025SeeU: Seeing the Unseen World via 4D Dynamics-aware Generationcited by 3

50 most cited of 87 All 87 (JSON)