# Learning to Orient Surfaces by Self-supervised Spherical CNNs

arXiv: https://arxiv.org/abs/2011.03298 · HTML: https://mlsanity.com/paper/2011.03298/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2020.

## Built on by (most cited first, 26 of 26)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors](https://mlsanity.com/paper/2109.00182/) (2021)
- [ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes](https://mlsanity.com/paper/2201.07788/) (2022)
- [The Devil is in the Pose: Ambiguity-free 3D Rotation-invariant Learning via Pose-aware Convolution](https://mlsanity.com/paper/2205.15210/) (2022)
- [Shape-Pose Disentanglement using SE(3)-equivariant Vector Neurons](https://mlsanity.com/paper/2204.01159/) (2022)
- [Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariance](https://mlsanity.com/paper/2304.10528/) (2023)
- [Self-Distillation for Unsupervised 3D Domain Adaptation](https://mlsanity.com/paper/2210.08226/) (2022)
- [Localization with Sampling-Argmax](https://mlsanity.com/paper/2110.08825/) (2021)
- [Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fields](https://mlsanity.com/paper/2212.02493/) (2022)
- [RefRec: Pseudo-labels Refinement via Shape Reconstruction for Unsupervised 3D Domain Adaptation](https://mlsanity.com/paper/2110.11036/) (2021)
- [SCARP: 3D Shape Completion in ARbitrary Poses for Improved Grasping](https://mlsanity.com/paper/2301.07213/) (2023)
- [WSDesc: Weakly Supervised 3D Local Descriptor Learning for Point Cloud Registration](https://mlsanity.com/paper/2108.02740/) (2021)
- [Rethinking Rotation Invariance with Point Cloud Registration](https://mlsanity.com/paper/2301.00149/) (2022)
- [Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes](https://mlsanity.com/paper/2102.01161/) (2021)
- [CRIN: Rotation-Invariant Point Cloud Analysis and Rotation Estimation via Centrifugal Reference Frame](https://mlsanity.com/paper/2303.03101/) (2023)
- [Open-Pose 3D Zero-Shot Learning: Benchmark and Challenges](https://mlsanity.com/paper/2312.07039/) (2023)
- [MaskLRF: Self-supervised Pretraining via Masked Autoencoding of Local Reference Frames for Rotation-invariant 3D Point Set Analysis](https://mlsanity.com/paper/2403.00206/) (2024)
- [Category-Level Global Camera Pose Estimation with Multi-Hypothesis Point Cloud Correspondences](https://mlsanity.com/paper/2209.14419/) (2022)
- [Stable and Consistent Prediction of 3D Characteristic Orientation via Invariant Residual Learning](https://mlsanity.com/paper/2306.11406/) (2023)
- [A Simple Strategy to Provable Invariance via Orbit Mapping](https://mlsanity.com/paper/2209.11916/) (2022)
- [TetraSphere: A Neural Descriptor for O(3)-Invariant Point Cloud Analysis](https://mlsanity.com/paper/2211.14456/) (2022)
- [General Rotation Invariance Learning for Point Clouds via Weight-Feature Alignment](https://mlsanity.com/paper/2302.09907/) (2023)
- [3D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Data](https://mlsanity.com/paper/2302.12883/) (2023)
- [Symmetry-Robust 3D Orientation Estimation](https://mlsanity.com/paper/2410.02101/) (2024)
- [Local Spherical Harmonics Improve Skeleton-Based Hand Action Recognition](https://mlsanity.com/paper/2308.10557/) (2023)
- [CANIS: Generation-Assisted 3D Canonicalization via an Image-Semantic Bridge](https://mlsanity.com/paper/2608.07256/) (2026)

## Abstract (arXiv)

Defining and reliably finding a canonical orientation for 3D surfaces is key to many Computer Vision and Robotics applications. This task is commonly addressed by handcrafted algorithms exploiting geometric cues deemed as distinctive and robust by the designer. Yet, one might conjecture that humans learn the notion of the inherent orientation of 3D objects from experience and that machines may do so alike. In this work, we show the feasibility of learning a robust canonical orientation for surfaces represented as point clouds. Based on the observation that the quintessential property of a canonical orientation is equivariance to 3D rotations, we propose to employ Spherical CNNs, a recently introduced machinery that can learn equivariant representations defined on the Special Orthogonal group SO(3). Specifically, spherical correlations compute feature maps whose elements define 3D rotations. Our method learns such feature maps from raw data by a self-supervised training procedure and robustly selects a rotation to transform the input point cloud into a learned canonical orientation. Thereby, we realize the first end-to-end learning approach to define and extract the canonical orientation of 3D shapes, which we aptly dub Compass. Experiments on several public datasets prove its effectiveness at orienting local surface patches as well as whole objects.