# Low-rank Bilinear Pooling for Fine-Grained Classification

arXiv: https://arxiv.org/abs/1611.05109 · HTML: https://mlsanity.com/paper/1611.05109/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2016.

## Built on by (most cited first, 30 of 42)

- [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021)
- [Fine-Grained Image Analysis with Deep Learning: A Survey](https://mlsanity.com/paper/2111.06119/) (2021)
- [Rank Diminishing in Deep Neural Networks](https://mlsanity.com/paper/2206.06072/) (2022)
- [Visual Attention Methods in Deep Learning: An In-Depth Survey](https://mlsanity.com/paper/2204.07756/) (2022)
- [Fine-Grained Object Classification via Self-Supervised Pose Alignment](https://mlsanity.com/paper/2203.15987/) (2022)
- [Learning Gabor Texture Features for Fine-Grained Recognition](https://mlsanity.com/paper/2308.05396/) (2023)
- [Improving Fine-Grained Visual Recognition in Low Data Regimes via Self-Boosting Attention Mechanism](https://mlsanity.com/paper/2208.00617/) (2022)
- [On the Eigenvalues of Global Covariance Pooling for Fine-grained Visual Recognition](https://mlsanity.com/paper/2205.13282/) (2022)
- [Bilinear value networks](https://mlsanity.com/paper/2204.13695/) (2022)
- [Tuning Pre-trained Model via Moment Probing](https://mlsanity.com/paper/2307.11342/) (2023)
- [Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry](https://mlsanity.com/paper/2407.10484/) (2024)
- [Object-aware Long-short-range Spatial Alignment for Few-Shot Fine-Grained Image Classification](https://mlsanity.com/paper/2108.13098/) (2021)
- [Part-guided Relational Transformers for Fine-grained Visual Recognition](https://mlsanity.com/paper/2212.13685/) (2022)
- [Learning Partial Correlation based Deep Visual Representation for Image Classification](https://mlsanity.com/paper/2304.11597/) (2023)
- [Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition](https://mlsanity.com/paper/2305.07180/) (2023)
- [Co-Salient Object Detection with Semantic-Level Consensus Extraction and Dispersion](https://mlsanity.com/paper/2309.07753/) (2023)
- [Progressive Co-Attention Network for Fine-grained Visual Classification](https://mlsanity.com/paper/2101.08527/) (2021)
- [Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2204.01707/) (2022)
- [Convolutional Fine-Grained Classification with Self-Supervised Target Relation Regularization](https://mlsanity.com/paper/2208.01997/) (2022)
- [Fine-grained Activities of People Worldwide](https://mlsanity.com/paper/2207.05182/) (2022)
- [Characterizing the Accuracy -- Efficiency Trade-off of Low-rank Decomposition in Language Models](https://mlsanity.com/paper/2405.06626/) (2024)
- [Attention to the Burstiness in Visual Prompt Tuning!](https://mlsanity.com/paper/2506.22908/) (2025)
- [Enhancing Fine-Grained Classification for Low Resolution Images](https://mlsanity.com/paper/2105.00241/) (2021)
- [Variational Stacked Local Attention Networks for Diverse Video Captioning](https://mlsanity.com/paper/2201.00985/) (2022)
- [Feature Boosting, Suppression, and Diversification for Fine-Grained Visual Classification](https://mlsanity.com/paper/2103.02782/) (2021)
- [Full-attention based Neural Architecture Search using Context Auto-regression](https://mlsanity.com/paper/2111.07139/) (2021)
- [Cross-layer Attention Network for Fine-grained Visual Categorization](https://mlsanity.com/paper/2210.08784/) (2022)
- [Parsing Objects at a Finer Granularity: A Survey](https://mlsanity.com/paper/2212.13693/) (2022)
- [Compositional Fine-Grained Low-Shot Learning](https://mlsanity.com/paper/2105.10438/) (2021)
- [Improved Benthic Classification using Resolution Scaling and SymmNet Unsupervised Domain Adaptation](https://mlsanity.com/paper/2303.10960/) (2023)

## Abstract (arXiv)

Pooling second-order local feature statistics to form a high-dimensional bilinear feature has been shown to achieve state-of-the-art performance on a variety of fine-grained classification tasks. To address the computational demands of high feature dimensionality, we propose to represent the covariance features as a matrix and apply a low-rank bilinear classifier. The resulting classifier can be evaluated without explicitly computing the bilinear feature map which allows for a large reduction in the compute time as well as decreasing the effective number of parameters to be learned. To further compress the model, we propose classifier co-decomposition that factorizes the collection of bilinear classifiers into a common factor and compact per-class terms. The co-decomposition idea can be deployed through two convolutional layers and trained in an end-to-end architecture. We suggest a simple yet effective initialization that avoids explicitly first training and factorizing the larger bilinear classifiers. Through extensive experiments, we show that our model achieves state-of-the-art performance on several public datasets for fine-grained classification trained with only category labels. Importantly, our final model is an order of magnitude smaller than the recently proposed compact bilinear model, and three orders smaller than the standard bilinear CNN model.