HiVLA: A Visual-Grounded-Centric Hierarchical Embodied Manipulation System
On the map: LLM reasoning → Spatial and embodied reasoning
also in 3 more
Abstract (arXiv)
While end-to-end Vision-Language-Action (VLA) models offer a promising paradigm for robotic manipulation, fine-tuning them on narrow control data often compromises the profound reasoning capabilities inherited from their base Vision-Language Models (VLMs). To resolve this fundamental trade-off, we propose HiVLA, a visual-grounded-centric hierarchical framework that explicitly decouples high-level semantic planning from low-level motor control. In high-level part, a VLM planner first performs task decomposition and visual grounding to generate structured plans, comprising a subtask instruction and a precise target bounding box. Then, to translate this plan into physical actions, we introduce a flow-matching Diffusion Transformer (DiT) action expert in low-level part equipped with a novel cascaded cross-attention mechanism. This design sequentially fuses global context, high-resolution object-centric crops and skill semantics, enabling the DiT to focus purely on robust execution. Our decoupled architecture preserves the VLM's zero-shot reasoning while allowing independent improvement of both components. Extensive experiments in simulation and the real world demonstrate that HiVLA significantly outperforms state-of-the-art end-to-end baselines, particularly excelling in long-horizon skill composition and the fine-grained manipulation of small objects in cluttered scenes.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 42 references that are in the map. First posted on arXiv 15 Apr 2026.
Builds on
38 of 42 references.
Closest ancestors · 3
Same direction, newest first
- 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting26 Sep
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models27 Mar
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success27 Feb
Other references · 35
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
23 more
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
- 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
- 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
- 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 216
- 2024RT-H: Action Hierarchies Using Languagecited by 204
- 2024PaliGemma 2: A Family of Versatile VLMs for Transfercited by 204
- 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
- 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
- 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 116
- 2025GR-3 Technical Reportcited by 110
- 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 103
- 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
- 2025InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policycited by 83
- 2025ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceivercited by 80
- 2024Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulationcited by 71
- 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
- 2025MemER: Scaling Up Memory for Robot Control via Experience Retrievalcited by 64
- 2025RoboGround: Robotic Manipulation with Grounded Vision-Language Priorscited by 50
- 2025Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructionscited by 50
- 2025H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulationcited by 41
- 20253D CAVLA: Leveraging Depth and 3D Context to Generalize Vision Language Action Models for Unseen Taskscited by 30
- 2025Scaling Vision Pre-Training to 4K Resolutioncited by 26
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routingcited by 41
- 2026ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-trainingcited by 2
- 2026World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulationcited by 1
- 2026SparkVLA: Stop-Aware Hierarchical VLA with Adaptive Action Chunking for Long-Horizon Manipulationcited by 1
- 2026GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulationcited by 0
- 2026H-VLA: Hierarchical Vision-Language-Action Model with Key-Action Reasoning and Motion Planning in a Unified Action Spacecited by 0
- 2026NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtimecited by 0
- 2026NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime7 Oct 2026
- 2026H-VLA: Hierarchical Vision-Language-Action Model with Key-Action Reasoning and Motion Planning in a Unified Action Space19 Sep 2026
- 2026ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training31 Aug 2026
- 2026GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulation27 Aug 2026
- 2026SparkVLA: Stop-Aware Hierarchical VLA with Adaptive Action Chunking for Long-Horizon Manipulation17 Aug 2026
- 2026World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulation5 Aug 2026
- 2026AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing8 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation
- 2025ManualVLA: A Unified VLA Model for Chain-of-Thought Manual Generation and Robotic Manipulation
- 2026Goal2Skill: Long-Horizon Manipulation with Adaptive Planning and Reflection
- 2025Unified Embodied VLM Reasoning with Robotic Action via Autoregressive Discretized Pre-training
- 2026Scaling World Model for Hierarchical Manipulation Policies
- 2026SG-VLA: Learning Spatially-Grounded Vision-Language-Action Models for Mobile Manipulation
- 2025Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey
- 2026Libra-VLA: Achieving Learning Equilibrium via Asynchronous Coarse-to-Fine Dual-System