# On the Eigenvalues of Global Covariance Pooling for Fine-grained Visual Recognition

arXiv: https://arxiv.org/abs/2205.13282 · HTML: https://mlsanity.com/paper/2205.13282/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 74 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 30 of 44)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [WebVision Database: Visual Learning and Understanding from Web Data](https://mlsanity.com/paper/1708.02862/) (2017)
- [Understanding Deep Networks via Extremal Perturbations and Smooth Masks](https://mlsanity.com/paper/1910.08485/) (2019)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [TransFG: A Transformer Architecture for Fine-grained Recognition](https://mlsanity.com/paper/2103.07976/) (2021)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches](https://mlsanity.com/paper/2003.03836/) (2020)
- [Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification from the Bottom Up](https://mlsanity.com/paper/1903.02827/) (2019)
- [Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization](https://mlsanity.com/paper/1712.01034/) (2017)
- [Part-Stacked CNN for Fine-Grained Visual Categorization](https://mlsanity.com/paper/1512.08086/) (2015)
- [There and Back Again: Revisiting Backpropagation Saliency Methods](https://mlsanity.com/paper/2004.02866/) (2020)
- [Learning Deep Bilinear Transformation for Fine-grained Image Representation](https://mlsanity.com/paper/1911.03621/) (2019)
- [Bird Species Categorization Using Pose Normalized Deep Convolutional Nets](https://mlsanity.com/paper/1406.2952/) (2014)
- [Hierarchical Bilinear Pooling for Fine-Grained Visual Recognition](https://mlsanity.com/paper/1807.09915/) (2018)
- [Is Second-order Information Helpful for Large-scale Visual Recognition?](https://mlsanity.com/paper/1703.08050/) (2017)

## Built on by (most cited first, 10 of 10)

- [Riemannian Multinomial Logistics Regression for SPD Neural Networks](https://mlsanity.com/paper/2305.11288/) (2023)
- [Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry](https://mlsanity.com/paper/2407.10484/) (2024)
- [RMLR: Extending Multinomial Logistic Regression into General Geometries](https://mlsanity.com/paper/2409.19433/) (2024)
- [Learning Fine-grained Domain Generalization via Hyperbolic State Space Hallucination](https://mlsanity.com/paper/2504.08020/) (2025)
- [Orthogonal SVD Covariance Conditioning and Latent Disentanglement](https://mlsanity.com/paper/2212.05599/) (2022)
- [Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality](https://mlsanity.com/paper/2207.02119/) (2022)
- [A Novel Image Classification Framework Based on Variational Quantum Algorithms](https://mlsanity.com/paper/2312.07932/) (2023)
- [Batch-efficient EigenDecomposition for Small and Medium Matrices](https://mlsanity.com/paper/2207.04228/) (2022)
- [Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorization](https://mlsanity.com/paper/2401.08860/) (2024)
- [Orthogonal Polynomial Approximation for Matrix Log Normalization in Global Covariance Pooling](https://mlsanity.com/paper/2608.19021/) (2026)

## Abstract (arXiv)

The Fine-Grained Visual Categorization (FGVC) is challenging because the subtle inter-class variations are difficult to be captured. One notable research line uses the Global Covariance Pooling (GCP) layer to learn powerful representations with second-order statistics, which can effectively model inter-class differences. In our previous conference paper, we show that truncating small eigenvalues of the GCP covariance can attain smoother gradient and improve the performance on large-scale benchmarks. However, on fine-grained datasets, truncating the small eigenvalues would make the model fail to converge. This observation contradicts the common assumption that the small eigenvalues merely correspond to the noisy and unimportant information. Consequently, ignoring them should have little influence on the performance. To diagnose this peculiar behavior, we propose two attribution methods whose visualizations demonstrate that the seemingly unimportant small eigenvalues are crucial as they are in charge of extracting the discriminative class-specific features. Inspired by this observation, we propose a network branch dedicated to magnifying the importance of small eigenvalues. Without introducing any additional parameters, this branch simply amplifies the small eigenvalues and achieves state-of-the-art performances of GCP methods on three fine-grained benchmarks. Furthermore, the performance is also competitive against other FGVC approaches on larger datasets. Code is available at hrefhttps://github.com/KingJamesSong/DifferentiableSVDhttps://github.com/KingJamesSong/DifferentiableSVD.