# Take-A-Photo: 3D-to-2D Generative Pre-training of Point Cloud Models

arXiv: https://arxiv.org/abs/2307.14971 · HTML: https://mlsanity.com/paper/2307.14971/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 65 references that are in the map.
- First posted on arXiv 27 Jul 2023.

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)

## Built on by (most cited first, 25 of 25)

- [ShapeLLM: Universal 3D Object Understanding for Embodied Interaction](https://mlsanity.com/paper/2402.17766/) (2024)
- [Point Cloud Pre-training with Diffusion Models](https://mlsanity.com/paper/2311.14960/) (2023)
- [PCP-MAE: Learning to Predict Centers for Point Masked Autoencoders](https://mlsanity.com/paper/2408.08753/) (2024)
- [Bringing Masked Autoencoders Explicit Contrastive Properties for Point Cloud Self-Supervised Learning](https://mlsanity.com/paper/2407.05862/) (2024)
- [PointNet with KAN versus PointNet with MLP for 3D Classification and Segmentation of Point Sets](https://mlsanity.com/paper/2410.10084/) (2024)
- [3D CoCa: Contrastive Learners are 3D Captioners](https://mlsanity.com/paper/2504.09518/) (2025)
- [Shape2Scene: 3D Scene Representation Learning Through Pre-training on Shape Data](https://mlsanity.com/paper/2407.10200/) (2024)
- [Towards More Diverse and Challenging Pre-training for Point Cloud Learning: Self-Supervised Cross Reconstruction with Decoupled Views](https://mlsanity.com/paper/2509.01250/) (2025)
- [UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting](https://mlsanity.com/paper/2506.09952/) (2025)
- [Harnessing Text-to-Image Diffusion Models for Point Cloud Self-Supervised Learning](https://mlsanity.com/paper/2507.09102/) (2025)
- [3D CoCa v2: Contrastive Learners with Test-Time Search for Generalizable Spatial Intelligence](https://mlsanity.com/paper/2601.06496/) (2026)
- [Point Cloud Self-supervised Learning via 3D to Multi-view Masked Learner](https://mlsanity.com/paper/2311.10887/) (2023)
- [Pre-training Point Cloud Compact Model with Partial-aware Reconstruction](https://mlsanity.com/paper/2407.09344/) (2024)
- [Masked Clustering Prediction for Unsupervised Point Cloud Pre-training](https://mlsanity.com/paper/2508.08910/) (2025)
- [PointAlign: Feature-Level Alignment Regularization for 3D Vision-Language Models](https://mlsanity.com/paper/2603.00412/) (2026)
- [Triple Point Masking](https://mlsanity.com/paper/2409.17547/) (2024)
- [Pic@Point: Cross-Modal Learning by Local and Global Point-Picture Correspondence](https://mlsanity.com/paper/2410.09519/) (2024)
- [DAP-MAE: Domain-Adaptive Point Cloud Masked Autoencoder for Effective Cross-Domain Learning](https://mlsanity.com/paper/2510.21635/) (2025)
- [Rethinking Point Clouds as Sequences: A Causal Next-Token Predictive Learning Framework](https://mlsanity.com/paper/2605.17566/) (2026)
- [From Extrinsic to Intrinsic: Geodesic-Guided Representation Learning for 3D Geometric Data](https://mlsanity.com/paper/2606.02268/) (2026)
- [PointCG: Self-supervised Point Cloud Learning via Joint Completion and Generation](https://mlsanity.com/paper/2411.06041/) (2024)
- [Towards Training-Free Open-World Classification with 3D Generative Models](https://mlsanity.com/paper/2501.17547/) (2025)
- [PointVDP: Learning View-Dependent Projection by Fireworks Rays for 3D Point Cloud Segmentation](https://mlsanity.com/paper/2507.06618/) (2025)
- [Dual-Branch Center-Surrounding Contrast: Rethinking Contrastive Learning for 3D Point Clouds](https://mlsanity.com/paper/2512.08673/) (2025)
- [GAPrompt++: Multi-Granular Geometry-Aware Point Cloud Prompt for 3D Vision Model](https://mlsanity.com/paper/2609.19716/) (2026)

## Abstract (arXiv)

With the overwhelming trend of mask image modeling led by MAE, generative pre-training has shown a remarkable potential to boost the performance of fundamental models in 2D vision. However, in 3D vision, the over-reliance on Transformer-based backbones and the unordered nature of point clouds have restricted the further development of generative pre-training. In this paper, we propose a novel 3D-to-2D generative pre-training method that is adaptable to any point cloud model. We propose to generate view images from different instructed poses via the cross-attention mechanism as the pre-training scheme. Generating view images has more precise supervision than its point cloud counterpart, thus assisting 3D backbones to have a finer comprehension of the geometrical structure and stereoscopic relations of the point cloud. Experimental results have proved the superiority of our proposed 3D-to-2D generative pre-training over previous pre-training methods. Our method is also effective in boosting the performance of architecture-oriented approaches, achieving state-of-the-art performance when fine-tuning on ScanObjectNN classification and ShapeNetPart segmentation tasks. Code is available at https://github.com/wangzy22/TAP.