# MVTN: Multi-View Transformation Network for 3D Shape Recognition

arXiv: https://arxiv.org/abs/2011.13244 · HTML: https://mlsanity.com/paper/2011.13244/

## Key facts

- Cited by 140 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2020.

## Built on by (most cited first, 30 of 140)

- [PointCLIP: Point Cloud Understanding by CLIP](https://mlsanity.com/paper/2112.02413/) (2021)
- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework](https://mlsanity.com/paper/2202.07123/) (2022)
- [ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding](https://mlsanity.com/paper/2212.05171/) (2022)
- [RVT: Robotic View Transformer for 3D Object Manipulation](https://mlsanity.com/paper/2306.14896/) (2023)
- [MVImgNet: A Large-scale Dataset of Multi-view Images](https://mlsanity.com/paper/2303.06042/) (2023)
- [ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding](https://mlsanity.com/paper/2305.08275/) (2023)
- [PointMamba: A Simple State Space Model for Point Cloud Analysis](https://mlsanity.com/paper/2402.10739/) (2024)
- [Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining](https://mlsanity.com/paper/2302.02318/) (2023)
- [CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-training](https://mlsanity.com/paper/2210.01055/) (2022)
- [Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoders](https://mlsanity.com/paper/2212.06785/) (2022)
- [ShapeLLM: Universal 3D Object Understanding for Embodied Interaction](https://mlsanity.com/paper/2402.17766/) (2024)
- [RVT-2: Learning Precise Manipulation from Few Demonstrations](https://mlsanity.com/paper/2406.08545/) (2024)
- [Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?](https://mlsanity.com/paper/2212.08320/) (2022)
- [PointGPT: Auto-regressively Generative Pre-training from Point Clouds](https://mlsanity.com/paper/2305.11487/) (2023)
- [Surface Representation for Point Clouds](https://mlsanity.com/paper/2205.05740/) (2022)
- [P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting](https://mlsanity.com/paper/2208.02812/) (2022)
- [Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Models](https://mlsanity.com/paper/2304.07221/) (2023)
- [Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model](https://mlsanity.com/paper/2404.14966/) (2024)
- [Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis](https://mlsanity.com/paper/2403.01439/) (2024)
- [Neural Point Light Fields](https://mlsanity.com/paper/2112.01473/) (2021)
- [Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation](https://mlsanity.com/paper/2204.07548/) (2022)
- [SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation](https://mlsanity.com/paper/2502.13143/) (2025)
- [Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning](https://mlsanity.com/paper/2410.08114/) (2024)
- [Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders](https://mlsanity.com/paper/2312.10726/) (2023)
- [Self-positioning Point-based Transformer for Point Cloud Understanding](https://mlsanity.com/paper/2303.16450/) (2023)
- [VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views](https://mlsanity.com/paper/2304.04617/) (2023)
- [Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysis](https://mlsanity.com/paper/2303.08134/) (2023)
- [PointAD: Comprehending 3D Anomalies from Points and Pixels for Zero-shot 3D Anomaly Detection](https://mlsanity.com/paper/2410.00320/) (2024)
- [PoinTramba: A Hybrid Transformer-Mamba Framework for Point Cloud Analysis](https://mlsanity.com/paper/2405.15463/) (2024)

## Abstract (arXiv)

Multi-view projection methods have demonstrated their ability to reach state-of-the-art performance on 3D shape recognition. Those methods learn different ways to aggregate information from multiple views. However, the camera view-points for those views tend to be heuristically set and fixed for all shapes. To circumvent the lack of dynamism of current multi-view methods, we propose to learn those view-points. In particular, we introduce the Multi-View Transformation Network (MVTN) that regresses optimal view-points for 3D shape recognition, building upon advances in differentiable rendering. As a result, MVTN can be trained end-to-end along with any multi-view network for 3D shape classification. We integrate MVTN in a novel adaptive multi-view pipeline that can render either 3D meshes or point clouds. MVTN exhibits clear performance gains in the tasks of 3D shape classification and 3D shape retrieval without the need for extra training supervision. In these tasks, MVTN achieves state-of-the-art performance on ModelNet40, ShapeNet Core55, and the most recent and realistic ScanObjectNN dataset (up to 6% improvement). Interestingly, we also show that MVTN can provide network robustness against rotation and occlusion in the 3D domain. The code is available at https://github.com/ajhamdi/MVTN .