mlsanityData to 7 Oct 2026

Sparse Imagination for Efficient Visual World Model Planning

arXiv 2506.01392 ↗First posted 2 Jun 2025cs.RO cs.AI cs.CVcited by 7

On the map: World models → Embodied and robot world models

Abstract (arXiv)

World model based planning has significantly improved decision-making in complex environments by enabling agents to simulate future states and make informed choices. This computational burden is particularly restrictive in robotics, where resources are severely constrained. To address this limitation, we propose a Sparse Imagination for Efficient Visual World Model Planning, which enhances computational efficiency by reducing the number of tokens processed during forward prediction. Our method leverages a sparsely trained vision-based world model based on transformers with randomized grouped attention strategy, allowing the model to flexibly adjust the number of tokens processed based on the computational resource. By enabling sparse imagination during latent rollout, our approach significantly accelerates planning while maintaining high control fidelity. Experimental results demonstrate that sparse imagination preserves task performance while dramatically improving inference efficiency. This general technique for visual planning is applicable from simple test-time trajectory optimization to complex real-world tasks with the latest VLAs, enabling the deployment of world models in real-time scenarios.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 33 of its 49 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

33 of 49 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 32

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  8. 2018World Modelscited by 1,444
  9. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  10. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  11. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  12. 2023Mastering Diverse Domains through World Modelscited by 1,193
20 more
  1. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  2. 2022Token Merging: Your ViT But Fastercited by 995
  3. 2020Mastering Atari with Discrete World Modelscited by 990
  4. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  5. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  6. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  7. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  8. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  9. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
  10. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  11. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
  12. 2022Masked Visual Pre-training for Motor Controlcited by 246
  13. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  14. 2021IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformerscited by 127
  15. 2024AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulationcited by 52
  16. 2022The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancycited by 35
  17. 2022PatchDropout: Economizing Vision Transformers Using Patch Dropoutcited by 28
  18. 2024Learning Correlation Structures for Vision Transformerscited by 15
  19. 2024Agglomerative Token Clusteringcited by 13
  20. 2024Learning to Rank Patches for Unbiased Image Redundancy Reductioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2026FF-JEPA: Long-Horizon Planning in World Models with Latent Plannerscited by 6
  2. 2026DDP-WM: Disentangled Dynamics Prediction for Efficient World Modelscited by 5
  3. 2025Edge General Intelligence Through World Models and Agentic AI: Fundamentals, Solutions, and Challengescited by 3
  4. 2026Learning Visual Feature-Based World Models via Residual Latent Actioncited by 2
  5. 2025Deep Active Inference with Diffusion Policy and Multiple Timescale World Model for Real-World Exploration and Navigationcited by 1
  6. 2026Scope-WM: Scoped Computation for Efficient Visual World Modelscited by 1
  7. 2026Sparse Planning in Visual World Models via Cost Gradientscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination
  2. 2026The Planning Limits of Latent World Models
  3. 2026Self-Correcting VLA: Online Action Refinement via Sparse World Imagination
  4. 2026SWEET: Sparse World Modeling with Image Editing for Embodied Task Execution
  5. 2026GlanceWAM: Sparse Test-Time Imagination for World-Action Models
  6. 2026Beyond Dense Futures: World Models as Structured Planners for Robotic Manipulation
  7. 2026World Action Modeling with Progressive Visual Planning
  8. 2026WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models