# DexVLG: Dexterous Vision-Language-Grasp Model at Scale

arXiv: https://arxiv.org/abs/2507.02747 · HTML: https://mlsanity.com/paper/2507.02747/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Human-video and latent-action pretraining, one of 13 directions in Vision-language-action models.
- Builds on 53 of its 75 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Placed in

- Vision-language-action models / [Human-video and latent-action pretraining](https://mlsanity.com/vla/human-video-and-latent-action-pretraining/)

## Builds on (most cited first, 30 of 53)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks](https://mlsanity.com/paper/2311.06242/) (2023)
- [Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://mlsanity.com/paper/2310.11441/) (2023)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [Uni3D: Exploring Unified 3D Representation at Scale](https://mlsanity.com/paper/2310.06773/) (2023)
- [OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding](https://mlsanity.com/paper/2305.10764/) (2023)
- [Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining](https://mlsanity.com/paper/2302.02318/) (2023)
- [Hand-Object Contact Consistency Reasoning for Human Grasps Generation](https://mlsanity.com/paper/2104.03304/) (2021)
- [DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation](https://mlsanity.com/paper/2403.07788/) (2024)
- [AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System](https://mlsanity.com/paper/2307.04577/) (2023)
- [DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation](https://mlsanity.com/paper/2210.02697/) (2022)
- [ShapeLLM: Universal 3D Object Understanding for Embodied Interaction](https://mlsanity.com/paper/2402.17766/) (2024)
- [OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction](https://mlsanity.com/paper/2203.15709/) (2022)
- [UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy](https://mlsanity.com/paper/2303.00938/) (2023)
- [Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?](https://mlsanity.com/paper/2212.08320/) (2022)
- [PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models](https://mlsanity.com/paper/2212.01558/) (2022)
- [UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning](https://mlsanity.com/paper/2304.00464/) (2023)
- [SAMPart3D: Segment Any Part in 3D Objects](https://mlsanity.com/paper/2411.07184/) (2024)
- [Synthesizing Diverse and Physically Stable Grasps with Arbitrary Hand Structures using Differentiable Force Closure Estimator](https://mlsanity.com/paper/2104.09194/) (2021)
- [Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Search](https://mlsanity.com/paper/2205.02961/) (2022)

## Built on by (most cited first, 20 of 20)

- [DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge](https://mlsanity.com/paper/2507.04447/) (2025)
- [Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos](https://mlsanity.com/paper/2507.15597/) (2025)
- [Pure Vision Language Action (VLA) Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.19012/) (2025)
- [Spatial-Aware VLA Pretraining through Visual-Physical Alignment from Human Videos](https://mlsanity.com/paper/2512.13080/) (2025)
- [DexHiL: A Human-in-the-Loop Framework for Vision-Language-Action Model Post-Training in Dexterous Manipulation](https://mlsanity.com/paper/2603.09121/) (2026)
- [Universal Dexterous Functional Grasping via Demonstration-Editing Reinforcement Learning](https://mlsanity.com/paper/2512.13380/) (2025)
- [DextER: Language-driven Dexterous Grasp Generation with Embodied Reasoning](https://mlsanity.com/paper/2601.16046/) (2026)
- [AugVLA-3D: Depth-Driven Feature Augmentation for Vision-Language-Action Models](https://mlsanity.com/paper/2602.10698/) (2026)
- [ZeroDexGrasp: Zero-Shot Task-Oriented Dexterous Grasp Synthesis with Prompt-Based Multi-Stage Semantic Reasoning](https://mlsanity.com/paper/2511.13327/) (2025)
- [From Power to Precision: Learning Fine-grained Dexterity for Multi-fingered Robotic Hands](https://mlsanity.com/paper/2511.13710/) (2025)
- [UniFucGrasp: Human-Hand-Inspired Unified Functional Grasp Annotation Strategy and Dataset for Diverse Dexterous Hands](https://mlsanity.com/paper/2508.03339/) (2025)
- [BORA: Bridging Offline Reinforcement Learning and Online Residual Adaptation for Real-World Dexterous VLA Models](https://mlsanity.com/paper/2605.30226/) (2026)
- [EAGG: Embodiment-Aligned Grasp Generation via Geometry-Aware Graph Conditioning](https://mlsanity.com/paper/2606.18092/) (2026)
- [Adaptive Vision-Language Grasping via Composable Foundation Priors and Generalizable Grasp Synthesis](https://mlsanity.com/paper/2609.04096/) (2026)
- [Structural Action Transformer for 3D Dexterous Manipulation](https://mlsanity.com/paper/2603.03960/) (2026)
- [BLaDA: Bridging Language to Functional Dexterous Actions within 3DGS Fields](https://mlsanity.com/paper/2604.08410/) (2026)
- [SECOND-Grasp: Semantic Contact-guided Dexterous Grasping](https://mlsanity.com/paper/2605.13117/) (2026)
- [KPGrasp: Scalable Keypoint Flow Matching for Dexterous Grasp Generation](https://mlsanity.com/paper/2606.09314/) (2026)
- [WristMimic: Full-Body Humanoid Control with Wrist-Guided Manipulation](https://mlsanity.com/paper/2607.06438/) (2026)
- [UniPart: Towards Zero-shot Language-Grounded 3D Part Segmentation for Embodied Interaction](https://mlsanity.com/paper/2609.12898/) (2026)

## Abstract (arXiv)

As large models gain traction, vision-language-action (VLA) systems are enabling robots to tackle increasingly complex tasks. However, limited by the difficulty of data collection, progress has mainly focused on controlling simple gripper end-effectors. There is little research on functional grasping with large models for human-like dexterous hands. In this paper, we introduce DexVLG, a large Vision-Language-Grasp model for Dexterous grasp pose prediction aligned with language instructions using single-view RGBD input. To accomplish this, we generate a dataset of 170 million dexterous grasp poses mapped to semantic parts across 174,000 objects in simulation, paired with detailed part-level captions. This large-scale dataset, named DexGraspNet 3.0, is used to train a VLM and flow-matching-based pose head capable of producing instruction-aligned grasp poses for tabletop objects. To assess DexVLG's performance, we create benchmarks in physics-based simulations and conduct real-world experiments. Extensive testing demonstrates DexVLG's strong zero-shot generalization capabilities-achieving over 76% zero-shot execution success rate and state-of-the-art part-grasp accuracy in simulation-and successful part-aligned grasps on physical objects in real-world scenarios.