# Beyond Visual Grasping: Benchmarking Complex Grasping from Detection to Execution

arXiv: https://arxiv.org/abs/2607.14341 · HTML: https://mlsanity.com/paper/2607.14341/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 51 references that are in the map.
- First posted on arXiv 15 Jul 2026.

## Builds on (most cited first, 30 of 37)

- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots](https://mlsanity.com/paper/2406.02523/) (2024)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains](https://mlsanity.com/paper/2212.08333/) (2022)
- [VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks](https://mlsanity.com/paper/2412.18194/) (2024)
- [Open-World Object Manipulation using Pre-trained Vision-Language Models](https://mlsanity.com/paper/2303.00905/) (2023)
- [THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation](https://mlsanity.com/paper/2402.08191/) (2024)
- [GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data](https://mlsanity.com/paper/2505.03233/) (2025)
- [Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations](https://mlsanity.com/paper/1912.04344/) (2019)
- [Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols](https://mlsanity.com/paper/1502.03143/) (2015)
- [DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping](https://mlsanity.com/paper/2502.20900/) (2025)
- [Jacquard: A Large Scale Dataset for Robotic Grasp Detection](https://mlsanity.com/paper/1803.11469/) (2018)
- [FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learning](https://mlsanity.com/paper/2401.08553/) (2024)
- [GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Grasping](https://mlsanity.com/paper/2403.09637/) (2024)
- [Deep Learning Approaches to Grasp Synthesis: A Review](https://mlsanity.com/paper/2207.02556/) (2022)
- [Open-Vocabulary Affordance Detection in 3D Point Clouds](https://mlsanity.com/paper/2303.02401/) (2023)
- [Grasp-Anything: Large-scale Grasp Dataset from Foundation Models](https://mlsanity.com/paper/2309.09818/) (2023)
- [SuctionNet-1Billion: A Large-Scale Benchmark for Suction Grasping](https://mlsanity.com/paper/2103.12311/) (2021)
- [GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Training](https://mlsanity.com/paper/2507.13097/) (2025)
- [SemGrasp: Semantic Grasp Generation via Language Aligned Discretization](https://mlsanity.com/paper/2404.03590/) (2024)
- [Language-driven Grasp Detection](https://mlsanity.com/paper/2406.09489/) (2024)
- [OCRTOC: A Cloud-Based Competition and Benchmark for Robotic Grasping and Manipulation](https://mlsanity.com/paper/2104.11446/) (2021)
- [Multi-FinGAN: Generative Coarse-To-Fine Sampling of Multi-Finger Grasps](https://mlsanity.com/paper/2012.09696/) (2020)
- [AdaGrasp: Learning an Adaptive Gripper-Aware Grasping Policy](https://mlsanity.com/paper/2011.14206/) (2020)

## Abstract (arXiv)

Robust robotic grasping remains a fundamental challenge for complex real-world applications. Recent advances in large-scale models demonstrate promising capabilities for reasoning in robotic tasks. However, existing benchmarks for grasping primarily focus on isolated, visual-based grasp pose detection, failing to capture the complexity of grasping tasks that require multi-step reasoning and semantic understanding during execution. To address this gap, we propose GCA-Bench, a benchmark featuring challenging textitgrasping with complex action scenarios that involve both scene-level reasoning and semantic constraints. GCA-Bench enables the evaluation of recent large foundation models under the same settings. To demonstrate the effectiveness of our new benchmark, we implement a diverse set of baselines, ranging from traditional grasp detection pipelines to end-to-end learning methods. Empirical studies achieve success rates below 70% on complex grasping scenarios, underscoring critical limitations. In addition, we propose new evaluation metrics, analyze critical failure models, and provide insights to guide the development of more robust and generalizable grasping strategies.