DeltaVLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation
On the map: Vision-language-action models → World model-based VLA
Abstract (arXiv)
Recent vision-language-action (VLA) models have significantly advanced robotic manipulation by unifying perception, reasoning, and control. To achieve such integration, recent studies adopt a predictive paradigm that models future visual states or world knowledge to guide action generation. However, these models emphasize forecasting outcomes rather than reasoning about the underlying process of change, which is essential for determining how to act. To address this, we propose DeltaVLA, a prior-guided framework that models world-knowledge variations relative to an explicit current-world knowledge prior for action generation, rather than regressing absolute future world states. Specifically, 1) to construct the current world knowledge prior, we propose the Prior-Guided WorldKnowledge Extractor (PWKE). It extracts manipulable regions, spatial relations, and semantic cues from the visual input, guided by auxiliary heads and prior pseudo labels, thus reducing redundancy. 2) Building upon this, to represent how world knowledge evolves under actions, we introduce the Latent World Variation Quantization (LWVQ). It learns a discrete latent space via a VQ-VAE objective to encode world knowledge variations, shifting prediction from full modalities to compact latent. 3)Moreover, to mitigate interference during variation modeling, we design the Conditional Variation Attention (CV-Atten), whichpromotes disentangled learning and preserves the independence of knowledge representations. Extensive experiments on both simulated benchmarks and real-world robotic tasks demonstrate DeltaVLA achieves state-of-the-art performance while improving efficiency. Code and real-world execution videos are available at https://github.com/JiuTian-VL/DeltaVLA.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in World model-based VLA, one of 13 directions in Vision-language-action models. Builds on 74 of its 83 references that are in the map. First posted on arXiv 9 Mar 2026.
Builds on
74 of 83 references.
Closest ancestors · 2
Same direction, newest first
- 2025F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actions8 Sep
- 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge6 Jul
Other references · 72
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2023Segment Anythingcited by 8,158
- 2023Visual Instruction Tuningcited by 7,792
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2017Neural Discrete Representation Learningcited by 4,650
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
38 more
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2024Depth Anything V2cited by 1,385
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
- 2022A Generalist Agentcited by 802
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
- 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
- 2023CoTracker: It is Better to Track Togethercited by 516
- 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
- 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
- 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
- 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
- 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
- 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
- 2024RT-H: Action Hierarchies Using Languagecited by 204
- 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
- 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 159
- 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 146
- 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 131
- 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 131
- 2023Learning to Act without Actionscited by 116
- 2025ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Modelcited by 112
- 2025UP-VLA: A Unified Understanding and Prediction Model for Embodied Agentcited by 99
- 2023Detecting and Grounding Multi-Modal Media Manipulationcited by 98
- 2025PD-VLA: Accelerating Vision-Language-Action Model Integrated with Action Chunking via Parallel Decodingcited by 88
- 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
50 most cited of 72 All 72 (JSON)
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognitioncited by 6
- 2026RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioningcited by 2
- 2026Retrieving to Recover: Towards Incomplete Audio-Visual Question Answering via Semantic-consistent Purificationcited by 1
- 2026MAVIS: Multi-Agent Video Retrieval via Structured Video Understandingcited by 1
- 2026World Tokens: Enhancing Embodied Policies with Training-Time World Modelingcited by 1
- 2026GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMscited by 0
- 2026GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulationcited by 0
- 2026Learning to Use Imagination: Progress-Conditioned Future Utilization for World Action Modelscited by 0
- 2026SCULPT-VLA: Learning Structured Control through Staged Action Groundingcited by 0
- 2026SCULPT-VLA: Learning Structured Control through Staged Action Grounding20 Sep 2026
- 2026Learning to Use Imagination: Progress-Conditioned Future Utilization for World Action Models6 Sep 2026
- 2026GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulation27 Aug 2026
- 2026World Tokens: Enhancing Embodied Policies with Training-Time World Modeling10 Aug 2026
- 2026GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs15 Jul 2026
- 2026RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning26 Jun 2026
- 2026MAVIS: Multi-Agent Video Retrieval via Structured Video Understanding8 Jun 2026
- 2026AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognition14 Apr 2026
- 2026Retrieving to Recover: Towards Incomplete Audio-Visual Question Answering via Semantic-consistent Purification12 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026AWM-VLA: AlignedWorld Modeling for Efficient and Explainable Vision-Language-Action Policies
- 2026SLIP-VLA: Single-Step Latent Imagination for Policy Learning in Vision-Language-Action Models
- 2026Being-H0.7: A Latent World-Action Model from Egocentric Videos
- 2026UniWAM: Unified World-Action Model
- 2026World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems
- 2026PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models
- 2026Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation
- 2026World Pilot: Steering Vision-Language-Action Models with World-Action Priors