# Learning to Grasp on the Moon from 3D Octree Observations with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2208.00818 · HTML: https://mlsanity.com/paper/2208.00818/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 1 Aug 2022.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Critics](https://mlsanity.com/paper/2005.04269/) (2020)
- [A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms](https://mlsanity.com/paper/1907.03146/) (2019)
- [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018)
- [Data-efficient Deep Reinforcement Learning for Dexterous Manipulation](https://mlsanity.com/paper/1704.03073/) (2017)
- [Gym-Ignition: Reproducible Robotic Simulations for Reinforcement Learning](https://mlsanity.com/paper/1911.01715/) (2019)
- [Pick and Place Without Geometric Object Models](https://mlsanity.com/paper/1707.05615/) (2017)
- [OctoPath: An OcTree Based Self-Supervised Learning Approach to Local Trajectory Planning for Mobile Robots](https://mlsanity.com/paper/2106.00988/) (2021)

## Built on by (most cited first, 6 of 6)

- [Space Robotics Bench: Robot Learning Beyond Earth](https://mlsanity.com/paper/2509.23328/) (2025)
- [OmniLRS: A Photorealistic Simulator for Lunar Robotics](https://mlsanity.com/paper/2309.08997/) (2023)
- [Autonomous Robotic Arm Manipulation for Planetary Missions using Causal Machine Learning](https://mlsanity.com/paper/2403.00470/) (2024)
- [LuSeg: Efficient Negative and Positive Obstacles Segmentation via Contrast-Driven Multi-Modal Feature Fusion on the Lunar](https://mlsanity.com/paper/2503.11409/) (2025)
- [Compact Multi-Object Placement Using Adjacency-Aware Reinforcement Learning](https://mlsanity.com/paper/2404.10632/) (2024)
- [Leveraging Procedural Generation for Learning Autonomous Peg-in-Hole Assembly in Space](https://mlsanity.com/paper/2405.01134/) (2024)

## Abstract (arXiv)

Extraterrestrial rovers with a general-purpose robotic arm have many potential applications in lunar and planetary exploration. Introducing autonomy into such systems is desirable for increasing the time that rovers can spend gathering scientific data and collecting samples. This work investigates the applicability of deep reinforcement learning for vision-based robotic grasping of objects on the Moon. A novel simulation environment with procedurally-generated datasets is created to train agents under challenging conditions in unstructured scenes with uneven terrain and harsh illumination. A model-free off-policy actor-critic algorithm is then employed for end-to-end learning of a policy that directly maps compact octree observations to continuous actions in Cartesian space. Experimental evaluation indicates that 3D data representations enable more effective learning of manipulation skills when compared to traditionally used image-based observations. Domain randomization improves the generalization of learned policies to novel scenes with previously unseen objects and different illumination conditions. To this end, we demonstrate zero-shot sim-to-real transfer by evaluating trained agents on a real robot in a Moon-analogue facility.