mlsanityData to 7 Oct 2026

Learning Real-World Action-Video Dynamics with Heterogeneous Masked Autoregression

arXiv 2502.04296 ↗First posted 6 Feb 2025cs.RO cs.CV cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

We propose Heterogeneous Masked Autoregression (HMA) for modeling action-video dynamics to generate high-quality data and evaluation in scaling robot learning. Building interactive video world models and policies for robotics is difficult due to the challenge of handling diverse settings while maintaining computational efficiency to run in real time. HMA uses heterogeneous pre-training from observations and action sequences across different robotic embodiments, domains, and tasks. HMA uses masked autoregression to generate quantized or soft tokens for video predictions. ourshort achieves better visual fidelity and controllability than the previous robotic video generation models with 15 times faster speed in the real world. After post-training, this model can be used as a video simulator from low-level action inputs for evaluating policies and generating synthetic data. See this link https://liruiw.github.io/hma for more information.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Feb 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 9.

21 papers.

Sort:

By area:

  1. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  2. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  3. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 117
  4. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  5. 2025RoboScape: Physics-informed Embodied World Modelcited by 38
  6. 2026BridgeV2W: Bridging Video Generation Models to Embodied World Models via Embodiment Maskscited by 19
  7. 2025ORV: 4D Occupancy-centric Robot Video Generationcited by 16
  8. 2026PlayWorld: Learning Robot World Models from Autonomous Playcited by 16
  9. 2025A Step Toward World Models: A Survey on Robotic Manipulationcited by 15
  10. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  11. 2026OAT: Ordered Action Tokenizationcited by 14
  12. 2025H2R-Grounder: A Paired-Data-Free Paradigm for Translating Human Interaction Videos into Physically Grounded Robot Videoscited by 8
9 more
  1. 2025Robotic VLA Benefits from Joint Learning with Motion Image Diffusioncited by 7
  2. 2025Towards High-Consistency Embodied World Model with Multi-View Trajectory Videoscited by 6
  3. 2025Flexible Multitask Learning with Factorized Diffusion Policycited by 5
  4. 2026RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluationcited by 4
  5. 2026DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulationcited by 3
  6. 2026BWM: A Low-Cost High-Fidelity World Simulator for Robot Learningcited by 2
  7. 2025Localized Graph-Based Neural Dynamics Models for Terrain Manipulationcited by 1
  8. 2025Learning Generalizable Visuomotor Policy through Dynamics-Alignmentcited by 0
  9. 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Physical Autoregressive Model for Robotic Manipulation without Action Pretraining
  2. 2026GigaWorld-Policy-0.5: A Faster and Stronger WAM Empowered by AutoResearch
  3. 2026Mask World Model: Predicting What Matters for Robust Robot Policy Learning
  4. 2026HiFlow: Tokenization-Free Scale-Wise Autoregressive Policy Learning via Flow Matching
  5. 2026Causal World Modeling for Robot Control
  6. 2026LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models
  7. 2026TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models
  8. 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data