# Shallow Deep Learning Can Still Excel in Fine-Grained Few-Shot Learning

arXiv: https://arxiv.org/abs/2507.22041 · HTML: https://mlsanity.com/paper/2507.22041/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Builds on (most cited first, 25 of 25)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Few-Shot Learning with Graph Neural Networks](https://mlsanity.com/paper/1711.04043/) (2017)
- [Revisiting Local Descriptor based Image-to-Class Measure for Few-shot Learning](https://mlsanity.com/paper/1903.12290/) (2019)
- [Few-Shot Classification with Feature Map Reconstruction Networks](https://mlsanity.com/paper/2012.01506/) (2020)
- [DPGN: Distribution Propagation Graph Network for Few-shot Learning](https://mlsanity.com/paper/2003.14247/) (2020)
- [Relational Embedding for Few-Shot Classification](https://mlsanity.com/paper/2108.09666/) (2021)
- [Advances and Challenges in Meta-Learning: A Technical Review](https://mlsanity.com/paper/2307.04722/) (2023)
- [Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification](https://mlsanity.com/paper/2204.04567/) (2022)
- [DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning](https://mlsanity.com/paper/2003.06777/) (2020)
- [Low-Rank Pairwise Alignment Bilinear Network For Few-Shot Fine-Grained Image Classification](https://mlsanity.com/paper/1908.01313/) (2019)
- [BSNet: Bi-Similarity Network for Few-shot Fine-grained Image Classification](https://mlsanity.com/paper/2011.14311/) (2020)
- [Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning](https://mlsanity.com/paper/2203.09064/) (2022)
- [Variational Feature Disentangling for Fine-Grained Few-Shot Classification](https://mlsanity.com/paper/2010.03255/) (2020)
- [Mixture-based Feature Space Learning for Few-shot Image Classification](https://mlsanity.com/paper/2011.11872/) (2020)
- [AMU-Tuning: Effective Logit Bias for CLIP-based Few-shot Learning](https://mlsanity.com/paper/2404.08958/) (2024)
- [Task Discrepancy Maximization for Fine-grained Few-Shot Classification](https://mlsanity.com/paper/2207.01376/) (2022)
- [Context-Aware Meta-Learning](https://mlsanity.com/paper/2310.10971/) (2023)
- [Knowledge Graph Transfer Network for Few-Shot Recognition](https://mlsanity.com/paper/1911.09579/) (2019)
- [Simple Semantic-Aided Few-Shot Learning](https://mlsanity.com/paper/2311.18649/) (2023)
- [Bi-directional Feature Reconstruction Network for Fine-Grained Few-Shot Image Classification](https://mlsanity.com/paper/2211.17161/) (2022)

## Abstract (arXiv)

Deep learning has witnessed the extensive utilization across a wide spectrum of domains, including fine-grained few-shot learning (FGFSL) which heavily depends on deep backbones. Nonetheless, shallower deep backbones such as ConvNet-4, are not commonly preferred because they're prone to extract a larger quantity of non-abstract visual attributes. In this paper, we initially re-evaluate the relationship between network depth and the ability to fully encode few-shot instances, and delve into whether shallow deep architecture could effectuate comparable or superior performance to mainstream deep backbone. Fueled by the inspiration from vanilla ConvNet-4, we introduce a location-aware constellation network (LCN-4), equipped with a cutting-edge location-aware feature clustering module. This module can proficiently encoder and integrate spatial feature fusion, feature clustering, and recessive feature location, thereby significantly minimizing the overall loss. Specifically, we innovatively put forward a general grid position encoding compensation to effectively address the issue of positional information missing during the feature extraction process of specific ordinary convolutions. Additionally, we further propose a general frequency domain location embedding technique to offset for the location loss in clustering features. We have carried out validation procedures on three representative fine-grained few-shot benchmarks. Relevant experiments have established that LCN-4 notably outperforms the ConvNet-4 based State-of-the-Arts and achieves performance that is on par with or superior to most ResNet12-based methods, confirming the correctness of our conjecture.