mlsanityData to 14 Oct 2026

Grasp-Anything: Large-scale Grasp Dataset from Foundation Models

arXiv 2309.09818 ↗First posted 18 Sep 2023cs.RO cs.CVcited by 33

Not on the map. Why?

Abstract (arXiv)

Foundation models such as ChatGPT have made significant strides in robotic tasks due to their universal representation of real-world domains. In this paper, we leverage foundation models to tackle grasp detection, a persistent challenge in robotics with broad industrial applications. Despite numerous grasp datasets, their object diversity remains limited compared to real-world figures. Fortunately, foundation models possess an extensive repository of real-world knowledge, including objects we encounter in our daily lives. As a consequence, a promising solution to the limited representation in previous grasp datasets is to harness the universal knowledge embedded in these foundation models. We present Grasp-Anything, a new large-scale grasp dataset synthesized from foundation models to implement this solution. Grasp-Anything excels in diversity and magnitude, boasting 1M samples with text descriptions and more than 3M objects, surpassing prior datasets. Empirically, we show that Grasp-Anything successfully facilitates zero-shot grasp detection on vision-based tasks and real-world robotic experiments. Our dataset and code are available at https://grasp-anything-2023.github.io.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 52 of its 71 references that are in the map. First posted on arXiv 18 Sep 2023.

Builds on

52 of 71 references.

References · 52

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2023Segment Anythingcited by 8,234
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  6. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  7. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  8. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  9. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  10. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
  11. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  12. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
38 more
  1. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  2. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  3. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  4. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  5. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 501
  6. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  7. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  8. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 299
  9. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 264
  10. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 242
  11. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  12. 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 208
  13. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 177
  14. 2020A Review of Generalized Zero-Shot Learning Methodscited by 159
  15. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 120
  16. 2020ACRONYM: A Large-Scale Grasp Dataset Based on Simulationcited by 117
  17. 2013Deep Learning for Detecting Robotic Graspscited by 116
  18. 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 108
  19. 2019Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrationscited by 107
  20. 2022DALL-E-Bot: Introducing Web-Scale Diffusion Models to Roboticscited by 106
  21. 2018Jacquard: A Large Scale Dataset for Robotic Grasp Detectioncited by 87
  22. 2018PointNetGPD: Detecting Grasp Configurations from Point Setscited by 80
  23. 2022Deep Learning Approaches to Grasp Synthesis: A Reviewcited by 72
  24. 2020EGAD! an Evolved Grasping Analysis Dataset for diversity and reproducibility in robotic manipulationcited by 61
  25. 2019Antipodal Robotic Grasping using Generative Residual Convolutional Neural Networkcited by 59
  26. 2021CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulationcited by 49
  27. 2021End-to-end Trainable Deep Neural Network for Robotic Grasp Detection and Semantic Segmentation from RGBcited by 44
  28. 2023Open-Vocabulary Affordance Detection in 3D Point Cloudscited by 41
  29. 2023A Joint Modeling of Vision-Language-Action for Target-oriented Grasping in Cluttercited by 34
  30. 2018ROI-based Robotic Grasp Detection for Object Overlapping Scenescited by 31
  31. 2019A Billion Ways to Grasp: An Evaluation of Grasp Sampling Schemes on a Dense, Physics-based Grasp Data Setcited by 30
  32. 2022Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformerscited by 17
  33. 2022MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesiscited by 13
  34. 2022Machine Learning-based Framework for Optimally Solving the Analytical Inverse Kinematics for Redundant Manipulatorscited by 9
  35. 2023HabiCrowd: A High Performance Simulator for Crowd-Aware Visual Navigationcited by 9
  36. 2022Grasp Learning: Models, Methods, and Performancecited by 8
  37. 2022Singularity Avoidance with Application to Online Trajectory Optimization for Serial Manipulatorscited by 7
  38. 2020Lio -- A Personal Robot Assistant for Human-Robot Interaction and Care Applicationscited by 6

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 13; 2025: 13; 2026: 6.

33 papers.

Sort:

By area:

  1. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  2. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 35
  3. 2024Language-driven Grasp Detectioncited by 26
  4. 2024Reasoning Grasping via Multimodal Large Language Modelcited by 20
  5. 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policycited by 18
  6. 2024Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidancecited by 16
  7. 2025RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Graspingcited by 11
  8. 2023Language-driven Scene Synthesis using Multi-conditional Diffusion Modelcited by 10
  9. 2024Lightweight Language-driven Grasp Detection using Conditional Consistency Modelcited by 9
  10. 2024Generative AI Agents in Autonomous Machines: A Safety Perspectivecited by 7
  11. 2025Spatial RoboGrasp: Generalized Robotic Grasping Control Policycited by 6
  12. 2024Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applicationscited by 5
21 more
  1. 2024Speeding up 6-DoF Grasp Sampling with Quality-Diversitycited by 4
  2. 2024ManiBox: Enhancing Embodied Spatial Generalization via Scalable Simulation Data Generationscited by 4
  3. 2025RoboGrasp: A Universal Grasping Policy for Robust Robotic Controlcited by 4
  4. 2024GraspMamba: A Mamba-based Language-driven Grasp Detection Framework with Hierarchical Feature Learningcited by 3
  5. 2025Efficient Alignment of Unconditioned Action Prior for Language-conditioned Pick and Place in Cluttercited by 3
  6. 2025Humanoid Agent via Embodied Chain-of-Action Reasoning with Multimodal Foundation Models for Zero-Shot Loco-Manipulationcited by 3
  7. 2024Language-driven Grasp Detection with Mask-guided Attentioncited by 2
  8. 2024QDGset: A Large Scale Grasping Dataset Generated with Quality-Diversitycited by 2
  9. 2025VCoT-Grasp: Grasp Foundation Models with Visual Chain-of-Thought Reasoning for Language-driven Grasp Generationcited by 2
  10. 2025Language-Guided Grasp Detection with Coarse-to-Fine Learning for Robotic Manipulationcited by 1
  11. 2026RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulationcited by 1
  12. 20243D Feature Distillation with Object-Centric Priorscited by 0
  13. 2025SpikePingpong: Spike Vision-based Fast-Slow Pingpong Robot Systemcited by 0
  14. 2025Communication-Efficient Module-Wise Federated Learning for Grasp Pose Detection in Cluttered Environmentscited by 0
  15. 2025A Multi-Level Similarity Approach for Single-View Object Grasping: Matching, Planning, and Fine-Tuningcited by 0
  16. 2025XGrasp: Gripper-Aware Grasp Detection with Multi-Gripper Data Generationcited by 0
  17. 2025VLAD-Grasp: Zero-shot Grasp Detection via Vision-Language Modelscited by 0
  18. 2026Affordance-Based Manipulation Planning with Text Goals and Sim-to-Real Generalisation via Real-to-Sim Image Conversioncited by 0
  19. 2026Beyond Visual Grasping: Benchmarking Complex Grasping from Detection to Executioncited by 0
  20. 2026Speech2Grasp: Data-Efficient Transfer of Text-Conditioned Grasp Detection to Speech in Humanoid Robotscited by 0
  21. 2026Gripper-aware Vision Language Action Modelscited by 0