# Grasping Trajectory Optimization with Point Clouds

arXiv: https://arxiv.org/abs/2403.05466 · HTML: https://mlsanity.com/paper/2403.05466/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 8 Mar 2024.

## Builds on (most cited first, 17 of 17)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [6-DOF GraspNet: Variational Grasp Generation for Object Manipulation](https://mlsanity.com/paper/1905.10520/) (2019)
- [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](https://mlsanity.com/paper/2103.14127/) (2021)
- [SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusion](https://mlsanity.com/paper/2209.03855/) (2022)
- [Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols](https://mlsanity.com/paper/1502.03143/) (2015)
- [Continuous-Time Gaussian Process Motion Planning via Probabilistic Inference](https://mlsanity.com/paper/1707.07383/) (2017)
- [A Billion Ways to Grasp: An Evaluation of Grasp Sampling Schemes on a Dense, Physics-based Grasp Data Set](https://mlsanity.com/paper/1912.05604/) (2019)
- [Fast Marching Tree: a Fast Marching Sampling-Based Method for Optimal Motion Planning in Many Dimensions](https://mlsanity.com/paper/1306.3532/) (2013)
- [Mean Shift Mask Transformer for Unseen Object Instance Segmentation](https://mlsanity.com/paper/2211.11679/) (2022)
- [Manipulation Trajectory Optimization with Online Grasp Synthesis and Selection](https://mlsanity.com/paper/1911.10280/) (2019)
- [A New Approach to Time-Optimal Path Parameterization based on Reachability Analysis](https://mlsanity.com/paper/1707.07239/) (2017)
- [Trajectory Optimization for Manipulation of Deformable Objects: Assembly of Belt Drive Units](https://mlsanity.com/paper/2106.00898/) (2021)
- [SCENEREPLICA: Benchmarking Real-World Robot Manipulation by Creating Replicable Scenes](https://mlsanity.com/paper/2306.15620/) (2023)
- [OpTaS: An Optimization-based Task Specification Library for Trajectory Optimization and Model Predictive Control](https://mlsanity.com/paper/2301.13512/) (2023)
- [GOMP: Grasp-Optimized Motion Planning for Bin Picking](https://mlsanity.com/paper/2003.02401/) (2020)

## Built on by (most cited first, 2 of 2)

- [Collision Avoidance Metric for 3D Camera Evaluation](https://mlsanity.com/paper/2405.09755/) (2024)
- [SE(3) Neural Potential Fields for 6-DoF Trajectory Planning Directly from Images Without Explicit 3D Reconstruction](https://mlsanity.com/paper/2609.24864/) (2026)

## Abstract (arXiv)

We introduce a new trajectory optimization method for robotic grasping based on a point-cloud representation of robots and task spaces. In our method, robots are represented by 3D points on their link surfaces. The task space of a robot is represented by a point cloud that can be obtained from depth sensors. Using the point-cloud representation, goal reaching in grasping can be formulated as point matching, while collision avoidance can be efficiently achieved by querying the signed distance values of the robot points in the signed distance field of the scene points. Consequently, a constrained nonlinear optimization problem is formulated to solve the joint motion and grasp planning problem. The advantage of our method is that the point-cloud representation is general to be used with any robot in any environment. We demonstrate the effectiveness of our method by performing experiments on a tabletop scene and a shelf scene for grasping with a Fetch mobile manipulator and a Franka Panda arm. The project page is available at urlhttps://irvlutd.github.io/GraspTrajOpt