# Modality-Autoregressive World-Action Models

arXiv: https://arxiv.org/abs/2609.17524 · HTML: https://mlsanity.com/paper/2609.17524/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 15 Sep 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 16 of 16)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025)
- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [Fast-WAM: Do World Action Models Need Test-time Future Imagination?](https://mlsanity.com/paper/2603.16666/) (2026)
- [Cosmos 3: Omnimodal World Models for Physical AI](https://mlsanity.com/paper/2606.02800/) (2026)
- [ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning](https://mlsanity.com/paper/2602.11236/) (2026)
- [Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generation](https://mlsanity.com/paper/2602.11401/) (2026)
- [EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data](https://mlsanity.com/paper/2607.08436/) (2026)
- [WAM4D: Fast 4D World Action Model via Spatial Register Tokens](https://mlsanity.com/paper/2606.14048/) (2026)
- [Turning Video Models into Generalist Robot Policies](https://mlsanity.com/paper/2605.27817/) (2026)
- [ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shifts](https://mlsanity.com/paper/2607.28993/) (2026)
- [Flex-π: A Multi-Stream World-Action Model with Compute Flexibility](https://mlsanity.com/paper/2608.10860/) (2026)
- [3PoinTr: From Human Videos to Robot Policies with 3D Point-Track Plans](https://mlsanity.com/paper/2603.08485/) (2026)
- [Point Tracking Improves World Action Models](https://mlsanity.com/paper/2605.23856/) (2026)
- [MV-WAM: Manifold-Aware World Action Model with Value Augmentation](https://mlsanity.com/paper/2606.21088/) (2026)
- [Modality Forcing for Scalable Spatial Generation](https://mlsanity.com/paper/2606.13676/) (2026)

## Built on by (most cited first, 2 of 2)

- [PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics](https://mlsanity.com/paper/2609.19142/) (2026)
- [PreferenceFlow: Test-Time Guidance of Flow-Matching Robot Policies from Human Interventions](https://mlsanity.com/paper/2609.36872/) (2026)

## Abstract (arXiv)

World-action models (WAMs) jointly model future observations and actions, typically predicting the future as RGB images. Other visual modalities such as depth, pretrained visual features, and point tracks can more efficiently capture geometric, semantic, and motion features. However, how best to combine these modalities within WAMs remains an open question. We introduce ModAR, the first WAM to autoregressively denoise multiple future modalities before predicting actions. This allows each prediction to condition on previously generated modalities. We train from scratch to systematically study how training-data mixtures, predicted modalities, and WAM formulations affect performance. In our evaluations, WAMs benefit from predicting point tracks, DINO features, and depth maps, while additionally predicting future RGB does not provide a consistent benefit. We also find that ModAR's sequential generation outperforms existing WAM formulations, with the highest average success rate at all evaluated data scales. We also fine-tune the video-model-initialized WAM Flex-π on the same data; ModAR achieves a slightly higher observed average success rate (75% vs. 72%) while using approximately 20× fewer training FLOPs and no pretraining. On three real-world bimanual tasks, ModAR outperforms baselines and improves with human videos.