# MVT: Multi-view Vision Transformer for 3D Object Recognition

arXiv: https://arxiv.org/abs/2110.13083 · HTML: https://mlsanity.com/paper/2110.13083/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [Rethinking Spatial Dimensions of Vision Transformers](https://mlsanity.com/paper/2103.16302/) (2021)
- [Volumetric and Multi-View CNNs for Object Classification on 3D Data](https://mlsanity.com/paper/1604.03265/) (2016)
- [VV-Net: Voxel VAE Net with Group Convolutions for Point Cloud Segmentation](https://mlsanity.com/paper/1811.04337/) (2018)
- [Container: Context Aggregation Network](https://mlsanity.com/paper/2106.01401/) (2021)
- [GIFT: A Real-time and Scalable 3D Shape Search Engine](https://mlsanity.com/paper/1604.01879/) (2016)
- [Dominant Set Clustering and Pooling for Multi-View 3D Object Recognition](https://mlsanity.com/paper/1906.01592/) (2019)
- [Pairwise Decomposition of Image Sequences for Active Multi-View Recognition](https://mlsanity.com/paper/1605.08359/) (2016)
- [LP-3DCNN: Unveiling Local Phase in 3D Convolutional Neural Networks](https://mlsanity.com/paper/1904.03498/) (2019)
- [3D-A-Nets: 3D Deep Dense Descriptor for Volumetric Shapes with Adversarial Networks](https://mlsanity.com/paper/1711.10108/) (2017)

## Built on by (most cited first, 30 of 34)

- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [A Survey of Visual Transformers](https://mlsanity.com/paper/2111.06091/) (2021)
- [Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction](https://mlsanity.com/paper/2112.13528/) (2021)
- [How to Train Vision Transformer on Small-scale Datasets?](https://mlsanity.com/paper/2210.07240/) (2022)
- [Defending Backdoor Attacks on Vision Transformer via Patch Processing](https://mlsanity.com/paper/2206.12381/) (2022)
- [BOAT: Bilateral Local Attention Vision Transformer](https://mlsanity.com/paper/2201.13027/) (2022)
- [Transformers Improve Breast Cancer Diagnosis from Unregistered Multi-View Mammograms](https://mlsanity.com/paper/2206.10096/) (2022)
- [Group Multi-View Transformer for 3D Shape Analysis with Spatial Encoding](https://mlsanity.com/paper/2312.16477/) (2023)
- [Uni3D-MoE: Scalable Multimodal 3D Scene Understanding via Mixture of Experts](https://mlsanity.com/paper/2505.21079/) (2025)
- [An Energy-Based Prior for Generative Saliency](https://mlsanity.com/paper/2204.08803/) (2022)
- [Dynamic Visual Prompt Tuning for Parameter Efficient Transfer Learning](https://mlsanity.com/paper/2309.06123/) (2023)
- [RBFormer: Improve Adversarial Robustness of Transformer by Robust Bias](https://mlsanity.com/paper/2309.13245/) (2023)
- [Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey](https://mlsanity.com/paper/2402.12923/) (2024)
- [PEVA-Net: Prompt-Enhanced View Aggregation Network for Zero/Few-Shot Multi-View 3D Shape Recognition](https://mlsanity.com/paper/2404.19168/) (2024)
- [R2-MLP: Round-Roll MLP for Multi-View 3D Object Recognition](https://mlsanity.com/paper/2211.11085/) (2022)
- [VSFormer: Mining Correlations in Flexible View Set for Multi-view 3D Shape Understanding](https://mlsanity.com/paper/2409.09254/) (2024)
- [Reg3D: Reconstructive Geometry Instruction Tuning for 3D Scene Understanding](https://mlsanity.com/paper/2509.03635/) (2025)
- [Learning Sequential Contexts using Transformer for 3D Hand Pose Estimation](https://mlsanity.com/paper/2206.00171/) (2022)
- [Multi-View Active Fine-Grained Recognition](https://mlsanity.com/paper/2206.01153/) (2022)
- [Degenerate Swin to Win: Plain Window-based Transformer without Sophisticated Operations](https://mlsanity.com/paper/2211.14255/) (2022)
- [ViewFormer: View Set Attention for Multi-view 3D Shape Understanding](https://mlsanity.com/paper/2305.00161/) (2023)
- [S^2Former-OR: Single-Stage Bi-Modal Transformer for Scene Graph Generation in OR](https://mlsanity.com/paper/2402.14461/) (2024)
- [Beyond Viewpoint: Robust 3D Object Recognition under Arbitrary Views through Joint Multi-Part Representation](https://mlsanity.com/paper/2407.03842/) (2024)
- [Proto-FG3D: Prototype-based Interpretable Fine-Grained 3D Shape Classification](https://mlsanity.com/paper/2505.17666/) (2025)
- [NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embedding](https://mlsanity.com/paper/2507.20110/) (2025)
- [MvBody: Multi-View-Based Hybrid Transformer Using Optical 3D Body Scan for Explainable Cesarean Section Prediction](https://mlsanity.com/paper/2511.03212/) (2025)
- [Multiple View Performers for Shape Completion](https://mlsanity.com/paper/2209.06291/) (2022)
- [Self-accumulative Vision Transformer for Bone Age Assessment Using the Sauvegrain Method](https://mlsanity.com/paper/2303.16557/) (2023)
- [Random Token Fusion for Multi-View Medical Diagnosis](https://mlsanity.com/paper/2410.15847/) (2024)
- [Hierarchical mutual distillation for multi-view fusion: Learning from all possible view combinations](https://mlsanity.com/paper/2411.10077/) (2024)

## Abstract (arXiv)

Inspired by the great success achieved by CNN in image recognition, view-based methods applied CNNs to model the projected views for 3D object understanding and achieved excellent performance. Nevertheless, multi-view CNN models cannot model the communications between patches from different views, limiting its effectiveness in 3D object recognition. Inspired by the recent success gained by vision Transformer in image recognition, we propose a Multi-view Vision Transformer (MVT) for 3D object recognition. Since each patch feature in a Transformer block has a global reception field, it naturally achieves communications between patches from different views. Meanwhile, it takes much less inductive bias compared with its CNN counterparts. Considering both effectiveness and efficiency, we develop a global-local structure for our MVT. Our experiments on two public benchmarks, ModelNet40 and ModelNet10, demonstrate the competitive performance of our MVT.