# Sparse Spatial Transformers for Few-Shot Learning

arXiv: https://arxiv.org/abs/2109.12932 · HTML: https://mlsanity.com/paper/2109.12932/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 27 Sep 2021.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [A Baseline for Few-Shot Image Classification](https://mlsanity.com/paper/1909.02729/) (2019)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [Cross Attention Network for Few-shot Classification](https://mlsanity.com/paper/1910.07677/) (2019)
- [CrossTransformers: spatially-aware few-shot transfer](https://mlsanity.com/paper/2007.11498/) (2020)
- [Task-Agnostic Meta-Learning for Few-shot Learning](https://mlsanity.com/paper/1805.07722/) (2018)
- [Revisiting Local Descriptor based Image-to-Class Measure for Few-shot Learning](https://mlsanity.com/paper/1903.12290/) (2019)
- [Relational Embedding for Few-Shot Classification](https://mlsanity.com/paper/2108.09666/) (2021)
- [Pareto Self-Supervised Training for Few-Shot Learning](https://mlsanity.com/paper/2104.07841/) (2021)
- [On Episodes, Prototypical Networks, and Few-shot Learning](https://mlsanity.com/paper/2012.09831/) (2020)
- [Few-Shot Learning with Global Class Representations](https://mlsanity.com/paper/1908.05257/) (2019)
- [Rethinking Class Relations: Absolute-relative Supervised and Unsupervised Few-shot Learning](https://mlsanity.com/paper/2001.03919/) (2020)
- [LGM-Net: Learning to Generate Matching Networks for Few-Shot Learning](https://mlsanity.com/paper/1905.06331/) (2019)
- [Multi-scale Adaptive Task Attention Network for Few-Shot Learning](https://mlsanity.com/paper/2011.14479/) (2020)
- [DMN4: Few-shot Learning via Discriminative Mutual Nearest Neighbor Neural Network](https://mlsanity.com/paper/2103.08160/) (2021)
- [Multi-level Metric Learning for Few-shot Image Recognition](https://mlsanity.com/paper/2103.11383/) (2021)
- [Learning to Focus: Cascaded Feature Matching Network for Few-shot Image Recognition](https://mlsanity.com/paper/2101.05018/) (2021)

## Built on by (most cited first, 14 of 14)

- [UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation](https://mlsanity.com/paper/2406.01188/) (2024)
- [Training-free and Adaptive Sparse Attention for Efficient Long Video Generation](https://mlsanity.com/paper/2502.21079/) (2025)
- [Multimodal Multi-loss Fusion Network for Sentiment Analysis](https://mlsanity.com/paper/2308.00264/) (2023)
- [Context-enriched molecule representations improve few-shot drug discovery](https://mlsanity.com/paper/2305.09481/) (2023)
- [Mask-guided Vision Transformer (MG-ViT) for Few-Shot Learning](https://mlsanity.com/paper/2205.09995/) (2022)
- [Enhancing Few-Shot Image Classification through Learnable Multi-Scale Embedding and Attention Mechanisms](https://mlsanity.com/paper/2409.07989/) (2024)
- [Minimizing Risk Through Minimizing Model-Data Interaction: A Protocol For Relying on Proxy Tasks When Designing Child Sexual Abuse Imagery Detection Models](https://mlsanity.com/paper/2505.06621/) (2025)
- [Shaping Visual Representations with Attributes for Few-Shot Recognition](https://mlsanity.com/paper/2112.06398/) (2021)
- [Hint-Aug: Drawing Hints from Foundation Vision Transformers Towards Boosted Few-Shot Parameter-Efficient Tuning](https://mlsanity.com/paper/2304.12520/) (2023)
- [The Devil is in the Few Shots: Iterative Visual Knowledge Completion for Few-shot Learning](https://mlsanity.com/paper/2404.09778/) (2024)
- [Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work](https://mlsanity.com/paper/2306.01929/) (2023)
- [Exploration of Class Center for Fine-Grained Visual Classification](https://mlsanity.com/paper/2407.04243/) (2024)
- [LeCoT: revisiting network architecture for two-view correspondence pruning](https://mlsanity.com/paper/2511.07078/) (2025)
- [PMCE: Probabilistic Multi-Granularity Semantics with Caption-Guided Enhancement for Few-Shot Learning](https://mlsanity.com/paper/2601.14111/) (2026)

## Abstract (arXiv)

Learning from limited data is challenging because data scarcity leads to a poor generalization of the trained model. A classical global pooled representation will probably lose useful local information. Many few-shot learning methods have recently addressed this challenge using deep descriptors and learning a pixel-level metric. However, using deep descriptors as feature representations may lose image contextual information. Moreover, most of these methods independently address each class in the support set, which cannot sufficiently use discriminative information and task-specific embeddings. In this paper, we propose a novel transformer-based neural network architecture called sparse spatial transformers (SSFormers), which finds task-relevant features and suppresses task-irrelevant features. Particularly, we first divide each input image into several image patches of different sizes to obtain dense local features. These features retain contextual information while expressing local information. Then, a sparse spatial transformer layer is proposed to find spatial correspondence between the query image and the full support set to select task-relevant image patches and suppress task-irrelevant image patches. Finally, we propose using an image patch-matching module to calculate the distance between dense local representations, thus determining which category the query image belongs to in the support set. Extensive experiments on popular few-shot learning benchmarks demonstrate the superiority of our method over state-of-the-art methods. Our source code is available at urlhttps://github.com/chenhaoxing/ssformers.