# RGBGrasp: Image-based Object Grasping by Capturing Multiple Views during Robot Arm Movement with Neural Radiance Fields

arXiv: https://arxiv.org/abs/2311.16592 · HTML: https://mlsanity.com/paper/2311.16592/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2023.

## Built on by (most cited first, 13 of 13)

- [A Survey of Embodied Learning for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2408.11537/) (2024)
- [An Economic Framework for 6-DoF Grasp Detection](https://mlsanity.com/paper/2407.08366/) (2024)
- [A Self-Correcting Vision-Language-Action Model for Fast and Slow System Manipulation](https://mlsanity.com/paper/2405.17418/) (2024)
- [Neural Fields in Robotics: A Survey](https://mlsanity.com/paper/2410.20220/) (2024)
- [DriveGen3D: Boosting Feed-Forward Driving Scene Generation with Efficient Video Diffusion](https://mlsanity.com/paper/2510.15264/) (2025)
- [TransDiff: Diffusion-Based Method for Manipulating Transparent Objects Using a Single RGB-D Image](https://mlsanity.com/paper/2503.12779/) (2025)
- [NeuGrasp: Generalizable Neural Surface Reconstruction with Background Priors for Material-Agnostic Object Grasp Detection](https://mlsanity.com/paper/2503.03511/) (2025)
- [Monocular One-Shot Metric-Depth Alignment for RGB-Based Robot Grasping](https://mlsanity.com/paper/2506.17110/) (2025)
- [Foundation Feature-Driven Online End-Effector Pose Estimation: A Marker-Free and Learning-Free Approach](https://mlsanity.com/paper/2503.14051/) (2025)
- [Data Augmentation for NeRFs in the Low Data Limit](https://mlsanity.com/paper/2503.02092/) (2025)
- [Exploiting Radiance Fields for Grasp Generation on Novel Synthetic Views](https://mlsanity.com/paper/2505.11467/) (2025)
- [3D Reconstruction Techniques in the Manufacturing Domain: Applications, Research Opportunities and Use Cases](https://mlsanity.com/paper/2604.28064/) (2026)
- [PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation](https://mlsanity.com/paper/2606.26916/) (2026)

## Abstract (arXiv)

Robotic research encounters a significant hurdle when it comes to the intricate task of grasping objects that come in various shapes, materials, and textures. Unlike many prior investigations that heavily leaned on specialized point-cloud cameras or abundant RGB visual data to gather 3D insights for object-grasping missions, this paper introduces a pioneering approach called RGBGrasp. This method depends on a limited set of RGB views to perceive the 3D surroundings containing transparent and specular objects and achieve accurate grasping. Our method utilizes pre-trained depth prediction models to establish geometry constraints, enabling precise 3D structure estimation, even under limited view conditions. Finally, we integrate hash encoding and a proposal sampler strategy to significantly accelerate the 3D reconstruction process. These innovations significantly enhance the adaptability and effectiveness of our algorithm in real-world scenarios. Through comprehensive experimental validations, we demonstrate that RGBGrasp achieves remarkable success across a wide spectrum of object-grasping scenarios, establishing it as a promising solution for real-world robotic manipulation tasks. The demonstrations of our method can be found on: https://sites.google.com/view/rgbgrasp