# View Inter-Prediction GAN: Unsupervised Representation Learning for 3D Shapes by Learning Global Shape Memories to Support Local View Predictions

arXiv: https://arxiv.org/abs/1811.02744 · HTML: https://mlsanity.com/paper/1811.02744/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2018.

## Built on by (most cited first, 30 of 43)

- [Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training](https://mlsanity.com/paper/2205.14401/) (2022)
- [CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding](https://mlsanity.com/paper/2203.00680/) (2022)
- [Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds](https://mlsanity.com/paper/2109.00179/) (2021)
- [SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer](https://mlsanity.com/paper/2108.04444/) (2021)
- [Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoders](https://mlsanity.com/paper/2212.06785/) (2022)
- [Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors](https://mlsanity.com/paper/2204.10603/) (2022)
- [Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2202.13589/) (2022)
- [Surface Reconstruction from Point Clouds by Learning Predictive Context Priors](https://mlsanity.com/paper/2204.11015/) (2022)
- [Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-training](https://mlsanity.com/paper/2302.14007/) (2023)
- [Cycle4Completion: Unpaired Point Cloud Completion using Cycle Transformation with Missing Region Coding](https://mlsanity.com/paper/2103.07838/) (2021)
- [Snowflake Point Deconvolution for Point Cloud Completion and Generation with Skip-Transformer](https://mlsanity.com/paper/2202.09367/) (2022)
- [Masked Autoencoders in 3D Point Cloud Representation Learning](https://mlsanity.com/paper/2207.01545/) (2022)
- [Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching](https://mlsanity.com/paper/2108.03746/) (2021)
- [Latent Partition Implicit with Surface Codes for 3D Representation](https://mlsanity.com/paper/2207.08631/) (2022)
- [Distillation with Contrast is All You Need for Self-Supervised Point Cloud Representation Learning](https://mlsanity.com/paper/2202.04241/) (2022)
- [3D-OAE: Occlusion Auto-Encoders for Self-Supervised Learning on Point Clouds](https://mlsanity.com/paper/2203.14084/) (2022)
- [POS-BERT: Point Cloud One-Stage BERT Pre-Training](https://mlsanity.com/paper/2204.00989/) (2022)
- [HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation](https://mlsanity.com/paper/2403.11817/) (2024)
- [PointMCD: Boosting Deep Point Cloud Encoders via Multi-view Cross-modal Distillation for 3D Shape Recognition](https://mlsanity.com/paper/2207.03128/) (2022)
- [ViPFormer: Efficient Vision-and-Pointcloud Transformer for Unsupervised Pointcloud Understanding](https://mlsanity.com/paper/2303.14376/) (2023)
- [Hierarchical View Predictor: Unsupervised 3D Global Feature Learning through Hierarchical Prediction among Unordered Views](https://mlsanity.com/paper/2108.03743/) (2021)
- [Representing Shape Collections with Alignment-Aware Linear Models](https://mlsanity.com/paper/2109.01605/) (2021)
- [PointVST: Self-Supervised Pre-training for 3D Point Clouds via View-Specific Point-to-Image Translation](https://mlsanity.com/paper/2212.14197/) (2022)
- [Lifelong Ensemble Learning based on Multiple Representations for Few-Shot Object Recognition](https://mlsanity.com/paper/2205.01982/) (2022)
- [SoftPool++: An Encoder-Decoder Network for Point Cloud Completion](https://mlsanity.com/paper/2205.03899/) (2022)
- [DHGCN: Dynamic Hop Graph Convolution Network for Self-Supervised Point Cloud Learning](https://mlsanity.com/paper/2401.02610/) (2024)
- [Point Discriminative Learning for Data-efficient 3D Point Cloud Analysis](https://mlsanity.com/paper/2108.02104/) (2021)
- [PointCMC: Cross-Modal Multi-Scale Correspondences Learning for Point Cloud Understanding](https://mlsanity.com/paper/2211.12032/) (2022)
- [Staying in Shape: Learning Invariant Shape Representations using Contrastive Learning](https://mlsanity.com/paper/2107.03552/) (2021)
- [Joint Data and Feature Augmentation for Self-Supervised Representation Learning on Point Clouds](https://mlsanity.com/paper/2211.01184/) (2022)

## Abstract (arXiv)

In this paper we present a novel unsupervised representation learning approach for 3D shapes, which is an important research challenge as it avoids the manual effort required for collecting supervised data. Our method trains an RNN-based neural network architecture to solve multiple view inter-prediction tasks for each shape. Given several nearby views of a shape, we define view inter-prediction as the task of predicting the center view between the input views, and reconstructing the input views in a low-level feature space. The key idea of our approach is to implement the shape representation as a shape-specific global memory that is shared between all local view inter-predictions for each shape. Intuitively, this memory enables the system to aggregate information that is useful to better solve the view inter-prediction tasks for each shape, and to leverage the memory as a view-independent shape representation. Our approach obtains the best results using a combination of L_2 and adversarial losses for the view inter-prediction task. We show that VIP-GAN outperforms state-of-the-art methods in unsupervised 3D feature learning on three large scale 3D shape benchmarks.