# Hierarchical View Predictor: Unsupervised 3D Global Feature Learning through Hierarchical Prediction among Unordered Views

arXiv: https://arxiv.org/abs/2108.03743 · HTML: https://mlsanity.com/paper/2108.03743/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 76 references that are in the map.
- First posted on arXiv 8 Aug 2021.

## Builds on (most cited first, 30 of 47)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [Volumetric and Multi-View CNNs for Object Classification on 3D Data](https://mlsanity.com/paper/1604.03265/) (2016)
- [DeepStereo: Learning to Predict New Views from the World's Imagery](https://mlsanity.com/paper/1506.06825/) (2015)
- [Learning a Predictable and Generative Vector Representation for Objects](https://mlsanity.com/paper/1603.08637/) (2016)
- [View Synthesis by Appearance Flow](https://mlsanity.com/paper/1605.03557/) (2016)
- [SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization](https://mlsanity.com/paper/1912.07109/) (2019)
- [Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces](https://mlsanity.com/paper/2011.13495/) (2020)
- [Generative and Discriminative Voxel Modeling with Convolutional Neural Networks](https://mlsanity.com/paper/1608.04236/) (2016)
- [Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision](https://mlsanity.com/paper/1612.00814/) (2016)
- [3D Point Capsule Networks](https://mlsanity.com/paper/1812.10775/) (2018)
- [Point Cloud Completion by Skip-attention Network with Hierarchical Folding](https://mlsanity.com/paper/2005.03871/) (2020)
- [PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention](https://mlsanity.com/paper/1810.05591/) (2018)
- [Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network](https://mlsanity.com/paper/1811.02565/) (2018)
- [PMP-Net: Point Cloud Completion by Learning Multi-step Point Moving Paths](https://mlsanity.com/paper/2012.03408/) (2020)
- [Multiresolution Tree Networks for 3D Point Cloud Processing](https://mlsanity.com/paper/1807.03520/) (2018)
- [Unsupervised Multi-Task Feature Learning on Point Clouds](https://mlsanity.com/paper/1910.08207/) (2019)
- [Point Cloud GAN](https://mlsanity.com/paper/1810.05795/) (2018)
- [Self-supervised Learning of Point Clouds via Orientation Estimation](https://mlsanity.com/paper/2008.00305/) (2020)
- [Triplet-Center Loss for Multi-View 3D Object Retrieval](https://mlsanity.com/paper/1803.06189/) (2018)
- [Unsupervised Learning of 3D Structure from Images](https://mlsanity.com/paper/1607.00662/) (2016)
- [Multi-Angle Point Cloud-VAE: Unsupervised Feature Learning for 3D Point Clouds from Multiple Angles by Joint Self-Reconstruction and Half-to-Half Prediction](https://mlsanity.com/paper/1907.12704/) (2019)
- [Learning Descriptor Networks for 3D Shape Synthesis and Analysis](https://mlsanity.com/paper/1804.00586/) (2018)
- [Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning](https://mlsanity.com/paper/2107.01886/) (2021)
- [VConv-DAE: Deep Volumetric Shape Learning Without Object Labels](https://mlsanity.com/paper/1604.03755/) (2016)
- [Cycle4Completion: Unpaired Point Cloud Completion using Cycle Transformation with Missing Region Coding](https://mlsanity.com/paper/2103.07838/) (2021)

## Built on by (most cited first, 7 of 7)

- [SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer](https://mlsanity.com/paper/2108.04444/) (2021)
- [Learning Deep Implicit Functions for 3D Shapes with Dynamic Code Clouds](https://mlsanity.com/paper/2203.14048/) (2022)
- [Snowflake Point Deconvolution for Point Cloud Completion and Generation with Skip-Transformer](https://mlsanity.com/paper/2202.09367/) (2022)
- [Fast Learning Radiance Fields by Shooting Much Fewer Rays](https://mlsanity.com/paper/2208.06821/) (2022)
- [Unsupervised Inference of Signed Distance Functions from Single Sparse Point Clouds without Learning Priors](https://mlsanity.com/paper/2303.14505/) (2023)
- [HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation](https://mlsanity.com/paper/2403.11817/) (2024)
- [Inferring Neural Signed Distance Functions by Overfitting on Single Noisy Point Clouds through Finetuning Data-Driven based Priors](https://mlsanity.com/paper/2410.19680/) (2024)

## Abstract (arXiv)

Unsupervised learning of global features for 3D shape analysis is an important research challenge because it avoids manual effort for supervised information collection. In this paper, we propose a view-based deep learning model called Hierarchical View Predictor (HVP) to learn 3D shape features from unordered views in an unsupervised manner. To mine highly discriminative information from unordered views, HVP performs a novel hierarchical view prediction over a view pair, and aggregates the knowledge learned from the predictions in all view pairs into a global feature. In a view pair, we pose hierarchical view prediction as the task of hierarchically predicting a set of image patches in a current view from its complementary set of patches, and in addition, completing the current view and its opposite from any one of the two sets of patches. Hierarchical prediction, in patches to patches, patches to view and view to view, facilitates HVP to effectively learn the structure of 3D shapes from the correlation between patches in the same view and the correlation between a pair of complementary views. In addition, the employed implicit aggregation over all view pairs enables HVP to learn global features from unordered views. Our results show that HVP can outperform state-of-the-art methods under large-scale 3D shape benchmarks in shape classification and retrieval.