# The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification

arXiv: https://arxiv.org/abs/2002.04264 · HTML: https://mlsanity.com/paper/2002.04264/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 30 of 64)

- [Fine-Grained Image Analysis with Deep Learning: A Survey](https://mlsanity.com/paper/2111.06119/) (2021)
- [CholecTriplet2021: A benchmark challenge for surgical action triplet recognition](https://mlsanity.com/paper/2204.04746/) (2022)
- [Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classification](https://mlsanity.com/paper/2101.06635/) (2021)
- [A Systematic Benchmarking Analysis of Transfer Learning for Medical Image Analysis](https://mlsanity.com/paper/2108.05930/) (2021)
- [Fine-Grained Object Classification via Self-Supervised Pose Alignment](https://mlsanity.com/paper/2203.15987/) (2022)
- [Domain Generalization via Frequency-domain-based Feature Disentanglement and Interaction](https://mlsanity.com/paper/2201.08029/) (2022)
- [SR-GNN: Spatial Relation-aware Graph Neural Network for Fine-Grained Image Categorization](https://mlsanity.com/paper/2209.02109/) (2022)
- [Fine-grained Recognition with Learnable Semantic Data Augmentation](https://mlsanity.com/paper/2309.00399/) (2023)
- [Dynamic MLP for Fine-Grained Image Classification by Leveraging Geographical and Temporal Information](https://mlsanity.com/paper/2203.03253/) (2022)
- [Adaptive Fine-Grained Predicates Learning for Scene Graph Generation](https://mlsanity.com/paper/2207.04602/) (2022)
- [Towards Fine-grained Visual Representations by Combining Contrastive Learning with Image Reconstruction and Attention-weighted Pooling](https://mlsanity.com/paper/2104.04323/) (2021)
- [Novel Class Discovery for Ultra-Fine-Grained Visual Categorization](https://mlsanity.com/paper/2405.06283/) (2024)
- [GrainSpace: A Large-scale Dataset for Fine-grained and Domain-adaptive Recognition of Cereal Grains](https://mlsanity.com/paper/2203.05306/) (2022)
- [R2-Trans:Fine-Grained Visual Categorization with Redundancy Reduction](https://mlsanity.com/paper/2204.10095/) (2022)
- [Fine-Grained Visual Classification using Self Assessment Classifier](https://mlsanity.com/paper/2205.10529/) (2022)
- [Part-guided Relational Transformers for Fine-grained Visual Recognition](https://mlsanity.com/paper/2212.13685/) (2022)
- [Take 5: Interpretable Image Classification with a Handful of Features](https://mlsanity.com/paper/2303.13166/) (2023)
- [Computer Vision on X-ray Data in Industrial Production and Security Applications: A Comprehensive Survey](https://mlsanity.com/paper/2211.05565/) (2022)
- [Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition](https://mlsanity.com/paper/2305.07180/) (2023)
- [Progressive Co-Attention Network for Fine-grained Visual Classification](https://mlsanity.com/paper/2101.08527/) (2021)
- [Convolutional Fine-Grained Classification with Self-Supervised Target Relation Regularization](https://mlsanity.com/paper/2208.01997/) (2022)
- [Channel DropBlock: An Improved Regularization Method for Fine-Grained Visual Classification](https://mlsanity.com/paper/2106.03432/) (2021)
- [Deep Class Incremental Learning from Decentralized Data](https://mlsanity.com/paper/2203.05984/) (2022)
- [Multi-Granularity Archaeological Dating of Chinese Bronze Dings Based on a Knowledge-Guided Relation Graph](https://mlsanity.com/paper/2303.15266/) (2023)
- [Coping with Change: Learning Invariant and Minimum Sufficient Representations for Fine-Grained Visual Categorization](https://mlsanity.com/paper/2306.04893/) (2023)
- [PETDet: Proposal Enhancement for Two-Stage Fine-Grained Object Detection](https://mlsanity.com/paper/2312.10515/) (2023)
- [Car-1000: A New Large Scale Fine-Grained Visual Categorization Dataset](https://mlsanity.com/paper/2503.12385/) (2025)
- [Interpretable Decision-Making for End-to-End Autonomous Driving](https://mlsanity.com/paper/2508.18898/) (2025)
- [Fine-Grained Visual Classification via Simultaneously Learning of Multi-regional Multi-grained Features](https://mlsanity.com/paper/2102.00367/) (2021)
- [A Task-aware Dual Similarity Network for Fine-grained Few-shot Learning](https://mlsanity.com/paper/2210.12348/) (2022)

## Abstract (arXiv)

Key for solving fine-grained image categorization is finding discriminate and local regions that correspond to subtle visual traits. Great strides have been made, with complex networks designed specifically to learn part-level discriminate feature representations. In this paper, we show it is possible to cultivate subtle details without the need for overly complicated network designs or training mechanisms -- a single loss is all it takes. The main trick lies with how we delve into individual feature channels early on, as opposed to the convention of starting from a consolidated feature map. The proposed loss function, termed as mutual-channel loss (MC-Loss), consists of two channel-specific components: a discriminality component and a diversity component. The discriminality component forces all feature channels belonging to the same class to be discriminative, through a novel channel-wise attention mechanism. The diversity component additionally constraints channels so that they become mutually exclusive on spatial-wise. The end result is therefore a set of feature channels that each reflects different locally discriminative regions for a specific class. The MC-Loss can be trained end-to-end, without the need for any bounding-box/part annotations, and yields highly discriminative regions during inference. Experimental results show our MC-Loss when implemented on top of common base networks can achieve state-of-the-art performance on all four fine-grained categorization datasets (CUB-Birds, FGVC-Aircraft, Flowers-102, and Stanford-Cars). Ablative studies further demonstrate the superiority of MC-Loss when compared with other recently proposed general-purpose losses for visual classification, on two different base networks. Code available at https://github.com/dongliangchang/Mutual-Channel-Loss