# Canonical Capsules: Self-Supervised Capsules in Canonical Pose

arXiv: https://arxiv.org/abs/2012.04718 · HTML: https://mlsanity.com/paper/2012.04718/

## Key facts

- Cited by 30 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 30 of 30)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [How to represent part-whole hierarchies in a neural network](https://mlsanity.com/paper/2102.12627/) (2021)
- [RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild](https://mlsanity.com/paper/2208.05963/) (2022)
- [Neural Kernel Surface Reconstruction](https://mlsanity.com/paper/2305.19590/) (2023)
- [You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors](https://mlsanity.com/paper/2109.00182/) (2021)
- [SQN: Weakly-Supervised Semantic Segmentation of Large-Scale 3D Point Clouds](https://mlsanity.com/paper/2104.04891/) (2021)
- [Neural Fields as Learnable Kernels for 3D Reconstruction](https://mlsanity.com/paper/2111.13674/) (2021)
- [GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds](https://mlsanity.com/paper/2305.16404/) (2023)
- [Equivariant Point Cloud Analysis via Learning Orientations for Message Passing](https://mlsanity.com/paper/2203.14486/) (2022)
- [ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes](https://mlsanity.com/paper/2201.07788/) (2022)
- [Pointersect: Neural Rendering with Cloud-Ray Intersection](https://mlsanity.com/paper/2304.12390/) (2023)
- [Self-supervised Learning for Pre-Training 3D Point Clouds: A Survey](https://mlsanity.com/paper/2305.04691/) (2023)
- [Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embedding](https://mlsanity.com/paper/2310.18716/) (2023)
- [Shape-Pose Disentanglement using SE(3)-equivariant Vector Neurons](https://mlsanity.com/paper/2204.01159/) (2022)
- [LiDAL: Inter-frame Uncertainty Based Active Learning for 3D LiDAR Semantic Segmentation](https://mlsanity.com/paper/2211.05997/) (2022)
- [Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Training](https://mlsanity.com/paper/2206.11215/) (2022)
- [Learning with Capsules: A Survey](https://mlsanity.com/paper/2206.02664/) (2022)
- [Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fields](https://mlsanity.com/paper/2212.02493/) (2022)
- [Robust Partial-to-Partial Point Cloud Registration in a Full Range](https://mlsanity.com/paper/2111.15606/) (2021)
- [LogoSP: Local-global Grouping of Superpoints for Unsupervised Semantic Segmentation of 3D Point Clouds](https://mlsanity.com/paper/2506.07857/) (2025)
- [ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformation](https://mlsanity.com/paper/2311.11106/) (2023)
- [A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Training](https://mlsanity.com/paper/2302.06019/) (2023)
- [Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes](https://mlsanity.com/paper/2102.01161/) (2021)
- [A Representation Separation Perspective to Correspondences-free Unsupervised 3D Point Cloud Registration](https://mlsanity.com/paper/2203.13239/) (2022)
- [Category-Level Global Camera Pose Estimation with Multi-Hypothesis Point Cloud Correspondences](https://mlsanity.com/paper/2209.14419/) (2022)
- [PointCaps: Raw Point Cloud Processing using Capsule Networks with Euclidean Distance Routing](https://mlsanity.com/paper/2112.11258/) (2021)
- [Unsupervised multi-branch Capsule for Hyperspectral and LiDAR classification](https://mlsanity.com/paper/2206.07348/) (2022)
- [Back to Square One: Superhuman Performance in Chutes and Ladders Through Deep Neural Networks and Tree Search](https://mlsanity.com/paper/2104.00698/) (2021)
- [CN-DHF: Compact Neural Double Height-Field Representations of 3D Shapes](https://mlsanity.com/paper/2304.13141/) (2023)

## Abstract (arXiv)

We propose a self-supervised capsule architecture for 3D point clouds. We compute capsule decompositions of objects through permutation-equivariant attention, and self-supervise the process by training with pairs of randomly rotated objects. Our key idea is to aggregate the attention masks into semantic keypoints, and use these to supervise a decomposition that satisfies the capsule invariance/equivariance properties. This not only enables the training of a semantically consistent decomposition, but also allows us to learn a canonicalization operation that enables object-centric reasoning. To train our neural network we require neither classification labels nor manually-aligned training datasets. Yet, by learning an object-centric representation in a self-supervised manner, our method outperforms the state-of-the-art on 3D point cloud reconstruction, canonicalization, and unsupervised classification.