# SVT-Net: Super Light-Weight Sparse Voxel Transformer for Large Scale Place Recognition

arXiv: https://arxiv.org/abs/2105.00149 · HTML: https://mlsanity.com/paper/2105.00149/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 1 May 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [Visual Transformers: Token-based Image Representation and Processing for Computer Vision](https://mlsanity.com/paper/2006.03677/) (2020)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition](https://mlsanity.com/paper/1804.03492/) (2018)
- [A2-Nets: Double Attention Networks](https://mlsanity.com/paper/1810.11579/) (2018)
- [MinkLoc3D: Point Cloud Based Large-Scale Place Recognition](https://mlsanity.com/paper/2011.04530/) (2020)
- [PCAN: 3D Attention Map Learning Using Contextual Information for Point Cloud Based Retrieval](https://mlsanity.com/paper/1904.09793/) (2019)
- [SOE-Net: A Self-Attention and Orientation Encoding Network for Point Cloud based Place Recognition](https://mlsanity.com/paper/2011.12430/) (2020)

## Built on by (most cited first, 30 of 36)

- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [R2Former: Unified Retrieval and Reranking Transformer for Place Recognition](https://mlsanity.com/paper/2304.03410/) (2023)
- [Bridging the Domain Gap for Multi-Agent Perception](https://mlsanity.com/paper/2210.08451/) (2022)
- [Improving Point Cloud Based Place Recognition with Ranking-based Loss and Large Batch Training](https://mlsanity.com/paper/2203.00972/) (2022)
- [Text2Loc: 3D Point Cloud Localization from Natural Language](https://mlsanity.com/paper/2311.15977/) (2023)
- [Transformers in 3D Point Clouds: A Survey](https://mlsanity.com/paper/2205.07417/) (2022)
- [General Place Recognition Survey: Towards Real-World Autonomy](https://mlsanity.com/paper/2405.04812/) (2024)
- [CASSPR: Cross Attention Single Scan Place Recognition](https://mlsanity.com/paper/2211.12542/) (2022)
- [Recent Advances in Vision Transformer: A Survey and Outlook of Recent Work](https://mlsanity.com/paper/2203.01536/) (2022)
- [CVTNet: A Cross-View Transformer Network for Place Recognition Using LiDAR Data](https://mlsanity.com/paper/2302.01665/) (2023)
- [AutoMerge: A Framework for Map Assembling and Smoothing in City-scale Environments](https://mlsanity.com/paper/2207.06965/) (2022)
- [CrossLoc3D: Aerial-Ground Cross-Source 3D Place Recognition](https://mlsanity.com/paper/2303.17778/) (2023)
- [CMMLoc: Advancing Text-to-PointCloud Localization with Cauchy-Mixture-Model Based Framework](https://mlsanity.com/paper/2503.02593/) (2025)
- [MSSPlace: Multi-Sensor Place Recognition with Visual and Text Semantics](https://mlsanity.com/paper/2407.15663/) (2024)
- [MinkUNeXt: Point Cloud-based Large-scale Place Recognition using 3D Sparse Convolutions](https://mlsanity.com/paper/2403.07593/) (2024)
- [ForestLPR: LiDAR Place Recognition in Forests Attentioning Multiple BEV Density Images](https://mlsanity.com/paper/2503.04475/) (2025)
- [RPR-Net: A Point Cloud-based Rotation-aware Large Scale Place Recognition Network](https://mlsanity.com/paper/2108.12790/) (2021)
- [VXP: Voxel-Cross-Pixel Large-scale Image-LiDAR Place Recognition](https://mlsanity.com/paper/2403.14594/) (2024)
- [Point Tree Transformer for Point Cloud Registration](https://mlsanity.com/paper/2406.17530/) (2024)
- [VLM-Loc: Localization in Point Cloud Maps via Vision-Language Models](https://mlsanity.com/paper/2603.09826/) (2026)
- [SAT: Size-Aware Transformer for 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2301.06869/) (2023)
- [Instance-free Text to Point Cloud Localization with Relative Position Awareness](https://mlsanity.com/paper/2404.17845/) (2024)
- [SelFLoc: Selective Feature Fusion for Large-scale Point Cloud-based Place Recognition](https://mlsanity.com/paper/2306.01205/) (2023)
- [Pose-Graph Attentional Graph Neural Network for Lidar Place Recognition](https://mlsanity.com/paper/2309.00168/) (2023)
- [PRISM-TopoMap: Online Topological Mapping with Place Recognition and Scan Matching](https://mlsanity.com/paper/2404.01674/) (2024)
- [L2RSI: Cross-view LiDAR-based Place Recognition for Large-scale Urban Scenes via Remote Sensing Imagery](https://mlsanity.com/paper/2503.11245/) (2025)
- [LifelongPR: Lifelong point cloud place recognition based on sample replay and prompt learning](https://mlsanity.com/paper/2507.10034/) (2025)
- [DSC: Deep Scan Context Descriptor for Large-Scale Place Recognition](https://mlsanity.com/paper/2111.13838/) (2021)
- [BPT: Binary Point Cloud Transformer for Place Recognition](https://mlsanity.com/paper/2303.01166/) (2023)
- [VNI-Net: Vector Neurons-based Rotation-Invariant Descriptor for LiDAR Place Recognition](https://mlsanity.com/paper/2308.12870/) (2023)

## Abstract (arXiv)

Point cloud-based large scale place recognition is fundamental for many applications like Simultaneous Localization and Mapping (SLAM). Although many models have been proposed and have achieved good performance by learning short-range local features, long-range contextual properties have often been neglected. Moreover, the model size has also become a bottleneck for their wide applications. To overcome these challenges, we propose a super light-weight network model termed SVT-Net for large scale place recognition. Specifically, on top of the highly efficient 3D Sparse Convolution (SP-Conv), an Atom-based Sparse Voxel Transformer (ASVT) and a Cluster-based Sparse Voxel Transformer (CSVT) are proposed to learn both short-range local features and long-range contextual features in this model. Consisting of ASVT and CSVT, SVT-Net can achieve state-of-the-art on benchmark datasets in terms of both accuracy and speed with a super-light model size (0.9M). Meanwhile, two simplified versions of SVT-Net are introduced, which also achieve state-of-the-art and further reduce the model size to 0.8M and 0.4M respectively.