# Fine-grained Classification via Categorical Memory Networks

arXiv: https://arxiv.org/abs/2012.06793 · HTML: https://mlsanity.com/paper/2012.06793/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2020.

## Built on by (most cited first, 3 of 3)

- [Context-Semantic Quality Awareness Network for Fine-Grained Visual Categorization](https://mlsanity.com/paper/2403.10298/) (2024)
- [Cached Transformers: Improving Transformers with Differentiable Memory Cache](https://mlsanity.com/paper/2312.12742/) (2023)
- [Multi-scale Activation, Refinement, and Aggregation: Exploring Diverse Cues for Fine-Grained Bird Recognition](https://mlsanity.com/paper/2504.09215/) (2025)

## Abstract (arXiv)

Motivated by the desire to exploit patterns shared across classes, we present a simple yet effective class-specific memory module for fine-grained feature learning. The memory module stores the prototypical feature representation for each category as a moving average. We hypothesize that the combination of similarities with respect to each category is itself a useful discriminative cue. To detect these similarities, we use attention as a querying mechanism. The attention scores with respect to each class prototype are used as weights to combine prototypes via weighted sum, producing a uniquely tailored response feature representation for a given input. The original and response features are combined to produce an augmented feature for classification. We integrate our class-specific memory module into a standard convolutional neural network, yielding a Categorical Memory Network. Our memory module significantly improves accuracy over baseline CNNs, achieving competitive accuracy with state-of-the-art methods on four benchmarks, including CUB-200-2011, Stanford Cars, FGVC Aircraft, and NABirds.