# MaskPlanner: Learning-Based Object-Centric Motion Generation from 3D Point Clouds

arXiv: https://arxiv.org/abs/2502.18745 · HTML: https://mlsanity.com/paper/2502.18745/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 79 references that are in the map.
- First posted on arXiv 26 Feb 2025.

## Builds on (most cited first, 30 of 39)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [Wayformer: Motion Forecasting via Simple & Efficient Attention Networks](https://mlsanity.com/paper/2207.05844/) (2022)
- [MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction](https://mlsanity.com/paper/2111.14973/) (2021)
- [Order Matters: Sequence to sequence for sets](https://mlsanity.com/paper/1511.06391/) (2015)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2005.08514/) (2020)
- [End-to-end people detection in crowded scenes](https://mlsanity.com/paper/1506.04878/) (2015)
- [ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queries](https://mlsanity.com/paper/2208.01582/) (2022)
- [Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion](https://mlsanity.com/paper/2111.12702/) (2021)
- [Learning to Paint With Model-based Deep Reinforcement Learning](https://mlsanity.com/paper/1903.04411/) (2019)
- [Robust Watertight Manifold Surface Generation Method for ShapeNet Models](https://mlsanity.com/paper/1802.01698/) (2018)
- [DeepGait: Planning and Control of Quadrupedal Gaits using Deep Reinforcement Learning](https://mlsanity.com/paper/1909.08399/) (2019)
- [From Perception to Decision: A Data-driven Approach to End-to-end Motion Planning for Autonomous Ground Robots](https://mlsanity.com/paper/1609.07910/) (2016)
- [Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environments](https://mlsanity.com/paper/1607.02329/) (2016)
- [Scalable Object Detection using Deep Neural Networks](https://mlsanity.com/paper/1312.2249/) (2013)
- [A survey on Deep Learning Advances on Different 3D Data Representations](https://mlsanity.com/paper/1808.01462/) (2018)
- [Neural Path Planning: Fixed Time, Near-Optimal Path Generation via Oracle Imitation](https://mlsanity.com/paper/1904.11102/) (2019)
- [Denoise and Contrast for Category Agnostic Shape Completion](https://mlsanity.com/paper/2103.16671/) (2021)
- [DeepSetNet: Predicting Sets with Deep Neural Networks](https://mlsanity.com/paper/1611.08998/) (2016)
- [3DSGrasp: 3D Shape-Completion for Robotic Grasp](https://mlsanity.com/paper/2301.00866/) (2023)

## Built on by (most cited first, 3 of 3)

- [3D-CovDiffusion: 3D-Aware Diffusion Policy for Coverage Path Planning](https://mlsanity.com/paper/2510.03011/) (2025)
- [MessyKitchens: Contact-rich object-level 3D scene reconstruction](https://mlsanity.com/paper/2603.16868/) (2026)
- [FoldPath: End-to-End Object-Centric Motion Generation via Modulated Implicit Paths](https://mlsanity.com/paper/2511.01407/) (2025)

## Abstract (arXiv)

Object-Centric Motion Generation (OCMG) plays a key role in a variety of industrial applicationsunicodex2014such as robotic spray painting and weldingunicodex2014requiring efficient, scalable, and generalizable algorithms to plan multiple long-horizon trajectories over free-form 3D objects. However, existing solutions rely on specialized heuristics, expensive optimization routines, or restrictive geometry assumptions that limit their adaptability to real-world scenarios. In this work, we introduce a novel, fully data-driven framework that tackles OCMG directly from 3D point clouds, learning to generalize expert path patterns across free-form surfaces. We propose MaskPlanner, a deep learning method that predicts local path segments for a given object while simultaneously inferring "path masks" to group these segments into distinct paths. This design induces the network to capture both local geometric patterns and global task requirements in a single forward pass. Extensive experimentation on a realistic robotic spray painting scenario shows that our approach attains near-complete coverage (above 99%) for unseen objects, while it remains task-agnostic and does not explicitly optimize for paint deposition. Moreover, our real-world validation on a 6-DoF specialized painting robot demonstrates that the generated trajectories are directly executable and yield expert-level painting quality. Our findings crucially highlight the potential of the proposed learning method for OCMG to reduce engineering overhead and seamlessly adapt to several industrial use cases.