# Graph-Fused Vision-Language-Action for Policy Reasoning in Multi-Arm Robotic Manipulation

arXiv: https://arxiv.org/abs/2509.07957 · HTML: https://mlsanity.com/paper/2509.07957/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in 3D and spatial-aware VLA, one of 13 directions in Vision-language-action models.
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 9 Sep 2025.

## Placed in

- Vision-language-action models / [3D and spatial-aware VLA](https://mlsanity.com/vla/3d-and-spatial-aware-vla/)

## Builds on (most cited first, 11 of 11)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [CLIPort: What and Where Pathways for Robotic Manipulation](https://mlsanity.com/paper/2109.12098/) (2021)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model](https://mlsanity.com/paper/2503.10631/) (2025)
- [InfoBot: Transfer and Exploration via the Information Bottleneck](https://mlsanity.com/paper/1901.10902/) (2019)
- [Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges](https://mlsanity.com/paper/2311.18044/) (2023)
- [HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation](https://mlsanity.com/paper/2406.14655/) (2024)
- [Exploiting Information Theory for Intuitive Robot Programming of Manual Activities](https://mlsanity.com/paper/2410.23963/) (2024)

## Abstract (arXiv)

Acquiring dexterous robotic skills from human video demonstrations remains a significant challenge, largely due to conventional reliance on low-level trajectory replication, which often fails to generalize across varying objects, spatial layouts, and manipulator configurations. To address this limitation, we introduce Graph-Fused Vision-Language-Action (GF-VLA), a unified framework that enables dual-arm robotic systems to perform task-level reasoning and execution directly from RGB-D human demonstrations. GF-VLA employs an information-theoretic approach to extract task-relevant cues, selectively highlighting critical hand-object and object-object interactions. These cues are structured into temporally ordered scene graphs, which are subsequently integrated with a language-conditioned transformer to produce hierarchical behavior trees and interpretable Cartesian motion primitives. To enhance efficiency in bimanual execution, we propose a cross-arm allocation strategy that autonomously determines gripper assignment without requiring explicit geometric modeling. We validate GF-VLA on four dual-arm block assembly benchmarks involving symbolic structure construction and spatial generalization. Empirical results demonstrate that the proposed representation achieves over 95% graph accuracy and 93% subtask segmentation, enabling the language-action planner to generate robust, interpretable task policies. When deployed on a dual-arm robot, these policies attain 94% grasp reliability, 89% placement accuracy, and 90% overall task success across stacking, letter-formation, and geometric reconfiguration tasks, evidencing strong generalization and robustness under diverse spatial and semantic variations.