# Group Multi-View Transformer for 3D Shape Analysis with Spatial Encoding

arXiv: https://arxiv.org/abs/2312.16477 · HTML: https://mlsanity.com/paper/2312.16477/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 62 references that are in the map.
- First posted on arXiv 27 Dec 2023.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework](https://mlsanity.com/paper/2202.07123/) (2022)
- [SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters](https://mlsanity.com/paper/1803.11527/) (2018)
- [MVTN: Multi-View Transformation Network for 3D Shape Recognition](https://mlsanity.com/paper/2011.13244/) (2020)
- [Generative and Discriminative Voxel Modeling with Convolutional Neural Networks](https://mlsanity.com/paper/1608.04236/) (2016)
- [MiniViT: Compressing Vision Transformers with Weight Multiplexing](https://mlsanity.com/paper/2204.07154/) (2022)
- [Adaptive Graph Convolution for Point Cloud Analysis](https://mlsanity.com/paper/2108.08035/) (2021)
- [PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition](https://mlsanity.com/paper/1808.07659/) (2018)
- [MVT: Multi-view Vision Transformer for 3D Object Recognition](https://mlsanity.com/paper/2110.13083/) (2021)
- [ViTKD: Practical Guidelines for ViT feature knowledge distillation](https://mlsanity.com/paper/2209.02432/) (2022)
- [GIFT: A Real-time and Scalable 3D Shape Search Engine](https://mlsanity.com/paper/1604.01879/) (2016)
- [Point2Vec for Self-Supervised Representation Learning on Point Clouds](https://mlsanity.com/paper/2303.16570/) (2023)
- [OccluMix: Towards De-Occlusion Virtual Try-on by Semantically-Guided Mixup](https://mlsanity.com/paper/2301.00965/) (2023)
- [Exploiting Local Geometry for Feature and Graph Construction for Better 3D Point Cloud Processing with Graph Neural Networks](https://mlsanity.com/paper/2103.15226/) (2021)
- [PVRNet: Point-View Relation Neural Network for 3D Shape Recognition](https://mlsanity.com/paper/1812.00333/) (2018)
- [R2-MLP: Round-Roll MLP for Multi-View 3D Object Recognition](https://mlsanity.com/paper/2211.11085/) (2022)
- [PREMA: Part-based REcurrent Multi-view Aggregation Network for 3D Shape Retrieval](https://mlsanity.com/paper/2111.04945/) (2021)

## Built on by (most cited first, 4 of 4)

- [Proto-FG3D: Prototype-based Interpretable Fine-Grained 3D Shape Classification](https://mlsanity.com/paper/2505.17666/) (2025)
- [CityGuard: Graph-Aware Private Descriptors for Bias-Resilient Identity Search Across Urban Cameras](https://mlsanity.com/paper/2602.18047/) (2026)
- [Set2Seq Transformer: Temporal and Position-Aware Set Representations for Sequential Multiple-Instance Learning](https://mlsanity.com/paper/2408.03404/) (2024)
- [PointCHR: Point Cloud Analysis via Curvature-Aware Hyperbolic Rectification](https://mlsanity.com/paper/2607.24052/) (2026)

## Abstract (arXiv)

In recent years, the results of view-based 3D shape recognition methods have saturated, and models with excellent performance cannot be deployed on memory-limited devices due to their huge size of parameters. To address this problem, we introduce a compression method based on knowledge distillation for this field, which largely reduces the number of parameters while preserving model performance as much as possible. Specifically, to enhance the capabilities of smaller models, we design a high-performing large model called Group Multi-view Vision Transformer (GMViT). In GMViT, the view-level ViT first establishes relationships between view-level features. Additionally, to capture deeper features, we employ the grouping module to enhance view-level features into group-level features. Finally, the group-level ViT aggregates group-level features into complete, well-formed 3D shape descriptors. Notably, in both ViTs, we introduce spatial encoding of camera coordinates as innovative position embeddings. Furthermore, we propose two compressed versions based on GMViT, namely GMViT-simple and GMViT-mini. To enhance the training effectiveness of the small models, we introduce a knowledge distillation method throughout the GMViT process, where the key outputs of each GMViT component serve as distillation targets. Extensive experiments demonstrate the efficacy of the proposed method. The large model GMViT achieves excellent 3D classification and retrieval results on the benchmark datasets ModelNet, ShapeNetCore55, and MCB. The smaller models, GMViT-simple and GMViT-mini, reduce the parameter size by 8 and 17.6 times, respectively, and improve shape recognition speed by 1.5 times on average, while preserving at least 90% of the classification and retrieval performance. The code is available at https://github.com/bigdata-graph/GMViT.