# A Deeper Look into Second-Order Feature Aggregation for LiDAR Place Recognition

arXiv: https://arxiv.org/abs/2409.15919 · HTML: https://mlsanity.com/paper/2409.15919/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 46 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Builds on (most cited first, 30 of 35)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition](https://mlsanity.com/paper/1804.03492/) (2018)
- [MixVPR: Feature Mixing for Visual Place Recognition](https://mlsanity.com/paper/2303.02190/) (2023)
- [Optimal Transport Aggregation for Visual Place Recognition](https://mlsanity.com/paper/2311.15937/) (2023)
- [MinkLoc3D: Point Cloud Based Large-Scale Place Recognition](https://mlsanity.com/paper/2011.04530/) (2020)
- [Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval](https://mlsanity.com/paper/2007.12163/) (2020)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [PCAN: 3D Attention Map Learning Using Contextual Information for Point Cloud Based Retrieval](https://mlsanity.com/paper/1904.09793/) (2019)
- [LCDNet: Deep Loop Closure Detection and Point Cloud Registration for LiDAR SLAM](https://mlsanity.com/paper/2103.05056/) (2021)
- [SOE-Net: A Self-Attention and Orientation Encoding Network for Point Cloud based Place Recognition](https://mlsanity.com/paper/2011.12430/) (2020)
- [Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization](https://mlsanity.com/paper/1712.01034/) (2017)
- [LoGG3D-Net: Locally Guided Global Descriptor Learning for 3D Place Recognition](https://mlsanity.com/paper/2109.08336/) (2021)
- [Improving Point Cloud Based Place Recognition with Ranking-based Loss and Large Batch Training](https://mlsanity.com/paper/2203.00972/) (2022)
- [Is Second-order Information Helpful for Large-scale Visual Recognition?](https://mlsanity.com/paper/1703.08050/) (2017)
- [Locus: LiDAR-based Place Recognition using Spatiotemporal Higher-Order Pooling](https://mlsanity.com/paper/2011.14497/) (2020)
- [NDT-Transformer: Large-Scale 3D Point Cloud Localisation using the Normal Distribution Transform Representation](https://mlsanity.com/paper/2103.12292/) (2021)
- [SVT-Net: Super Light-Weight Sparse Voxel Transformer for Large Scale Place Recognition](https://mlsanity.com/paper/2105.00149/) (2021)
- [AdaFusion: Visual-LiDAR Fusion with Adaptive Weights for Place Recognition](https://mlsanity.com/paper/2111.11739/) (2021)
- [Improved Bilinear Pooling with CNNs](https://mlsanity.com/paper/1707.06772/) (2017)
- [TransLoc3D : Point Cloud based Large-scale Place Recognition using Adaptive Receptive Fields](https://mlsanity.com/paper/2105.11605/) (2021)
- [Tensor Representations for Action Recognition](https://mlsanity.com/paper/2012.14371/) (2020)
- [Power Normalizations in Fine-grained Image, Few-shot Image and Graph Classification](https://mlsanity.com/paper/2012.13975/) (2020)
- [EgoNN: Egocentric Neural Network for Point Cloud Based 6DoF Relocalization at the City Scale](https://mlsanity.com/paper/2110.12486/) (2021)
- [Spectral Geometric Verification: Re-Ranking Point Cloud Retrieval for Metric Localization](https://mlsanity.com/paper/2210.04432/) (2022)
- [Factorized Bilinear Models for Image Recognition](https://mlsanity.com/paper/1611.05709/) (2016)

## Abstract (arXiv)

Efficient LiDAR Place Recognition (LPR) compresses dense pointwise features into compact global descriptors. While first-order aggregators such as GeM and NetVLAD are widely used, they overlook inter-feature correlations that second-order aggregation naturally captures. Full covariance, a common second-order aggregator, is high in dimensionality; as a result, practitioners often insert a learned projection or employ random sketches -- both of which either sacrifice information or increase parameter count. However, no prior work has systematically investigated how first- and second-order aggregation perform under constrained feature and compute budgets. In this paper, we first demonstrate that second-order aggregation retains its superiority for LPR even when channels are pruned and backbone parameters are reduced. Building on this insight, we propose Channel Partition-based Second-order Local Feature Aggregation (CPS): a drop-in, partition-based second-order aggregation module that preserves all channels while producing an order-of-magnitude smaller descriptor. CPS matches or exceeds the performance of full covariance and outperforms random projection variants, delivering new state-of-the-art results with only four additional learnable parameters across four large-scale benchmarks: Oxford RobotCar, In-house, MulRan, and WildPlaces.