mlsanityData to 7 Oct 2026

WorldDiT: A Unified Diffusion Architecture for World and Action Modeling

arXiv 2607.23909 ↗First posted 27 Jul 2026cs.LG cs.ROcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Many recent robot policies pursue stronger control by using large pretrained vision-language models (VLMs) as the action backbone. We introduce WorldDiT, a unified diffusion transformer architecture that couples action generation with visual world modeling and achieves strong performance without a large pretrained VLM action backbone. During training, a single diffusion transformer generates continuous action chunks and predicts normalized RGB patch targets from future camera frames. Across four LIBERO simulation suites, WorldDiT lies on the reported Pareto frontier for total model parameters and mean success among methods reporting all four suites. These results provide a strong sub-billion-parameter baseline for future scaling studies.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 27 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

27 of 27 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026VLANeXt: Recipes for Building Strong VLA Models20 Feb
  2. 2026ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models16 Jan
  3. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model11 Sep
  4. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models27 Mar
  5. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success27 Feb

Other references · 22

  1. 2022Flow Matching for Generative Modelingcited by 5,426
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  4. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  5. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  6. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  7. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  8. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  9. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  10. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  11. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  12. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
10 more
  1. 2025MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulationcited by 183
  2. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 159
  3. 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
  4. 2025FlowVLA: Visual Chain of Thought-based Motion Reasoning for Vision-Language-Action Modelscited by 50
  5. 2025VLA-0: Building State-of-the-Art VLAs with Zero Modificationcited by 49
  6. 2024Diffusion Transformer Policycited by 41
  7. 2025Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Processcited by 37
  8. 2026MMaDA-VLA: Large Diffusion Vision-Language-Action Model with Unified Multi-Modal Instruction and Generationcited by 10
  9. 2025Paris: A Decentralized Trained Open-Weight Diffusion Modelcited by 3
  10. 2026Paris 2.0: A Decentralized Diffusion Model for Video Generationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Spatially Aware World Action Model via Geometric Latent Diffusion
  2. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets
  3. 2026Causal World Modeling for Robot Control
  4. 2026AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing
  5. 2025RynnVLA-002: A Unified Vision-Language-Action and World Model
  6. 2026Mask World Model: Predicting What Matters for Robust Robot Policy Learning
  7. 2026World Action Models are Zero-shot Policies
  8. 2026World Tokens: Enhancing Embodied Policies with Training-Time World Modeling