# Robust Point Cloud Reinforcement Learning via PCA-Based Canonicalization

arXiv: https://arxiv.org/abs/2510.20974 · HTML: https://mlsanity.com/paper/2510.20974/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 30 of 31)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills](https://mlsanity.com/paper/2302.04659/) (2023)
- [Escaping the Big Data Paradigm with Compact Transformers](https://mlsanity.com/paper/2104.05704/) (2021)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [PointGPT: Auto-regressively Generative Pre-training from Point Clouds](https://mlsanity.com/paper/2305.11487/) (2023)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [Rotation Invariant Convolutions for 3D Point Clouds Deep Learning](https://mlsanity.com/paper/1908.06297/) (2019)
- [Learning Generalizable Dexterous Manipulation from Human Grasp Affordance](https://mlsanity.com/paper/2204.02320/) (2022)
- [Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning](https://mlsanity.com/paper/2111.03062/) (2021)
- [Rotation-Invariant Local-to-Global Representation Learning for 3D Point Cloud](https://mlsanity.com/paper/2010.03318/) (2020)
- [Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds](https://mlsanity.com/paper/2210.07442/) (2022)
- [Demonstrating Agile Flight from Pixels without State Estimation](https://mlsanity.com/paper/2406.12505/) (2024)
- [RIConv++: Effective Rotation Invariant Convolutions for 3D Point Clouds Deep Learning](https://mlsanity.com/paper/2202.13094/) (2022)
- [RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalization](https://mlsanity.com/paper/2307.10224/) (2023)
- [LapGym -- An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery](https://mlsanity.com/paper/2302.09606/) (2023)
- [Point Cloud Models Improve Visual Robustness in Robotic Learners](https://mlsanity.com/paper/2404.18926/) (2024)
- [Triangle-Net: Towards Robustness in Point Cloud Learning](https://mlsanity.com/paper/2003.00856/) (2020)
- [On the Efficacy of 3D Point Cloud Reinforcement Learning](https://mlsanity.com/paper/2306.06799/) (2023)
- [Adversarial Imitation Learning from Visual Observations using Latent Information](https://mlsanity.com/paper/2309.17371/) (2023)
- [Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learning](https://mlsanity.com/paper/2407.12792/) (2024)
- [PointPatchRL -- Masked Reconstruction Improves Reinforcement Learning on Point Clouds](https://mlsanity.com/paper/2410.18800/) (2024)
- [A general framework for rotation invariant point cloud analysis](https://mlsanity.com/paper/2402.01331/) (2024)

## Built on by (most cited first, 1 of 1)

- [On Preserving Geometrical Invariance for Superpixel Image Classification using Graph Transformer](https://mlsanity.com/paper/2607.04262/) (2026)

## Abstract (arXiv)

Reinforcement Learning (RL) from raw visual input has achieved impressive successes in recent years, yet it remains fragile to out-of-distribution variations such as changes in lighting, color, and viewpoint. Point Cloud Reinforcement Learning (PC-RL) offers a promising alternative by mitigating appearance-based brittleness, but its sensitivity to camera pose mismatches continues to undermine reliability in realistic settings. To address this challenge, we propose PCA Point Cloud (PPC), a canonicalization framework specifically tailored for downstream robotic control. PPC maps point clouds under arbitrary rigid-body transformations to a unique canonical pose, aligning observations to a consistent frame, thereby substantially decreasing viewpoint-induced inconsistencies. In our experiments, we show that PPC improves robustness to unseen camera poses across challenging robotic tasks, providing a principled alternative to domain randomization.