# Discriminative Feature Learning through Feature Distance Loss

arXiv: https://arxiv.org/abs/2205.11606 · HTML: https://mlsanity.com/paper/2205.11606/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Object Detectors Emerge in Deep Scene CNNs](https://mlsanity.com/paper/1412.6856/) (2014)
- [Richer Convolutional Features for Edge Detection](https://mlsanity.com/paper/1612.02103/) (2016)
- [Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrieval](https://mlsanity.com/paper/1604.04994/) (2016)
- [Feature Fusion for Online Mutual Knowledge Distillation](https://mlsanity.com/paper/1904.09058/) (2019)
- [Do semantic parts emerge in Convolutional Neural Networks?](https://mlsanity.com/paper/1607.03738/) (2016)
- [Parameter-Free Spatial Attention Network for Person Re-Identification](https://mlsanity.com/paper/1811.12150/) (2018)
- [Mask-CNN: Localizing Parts and Selecting Descriptors for Fine-Grained Image Recognition](https://mlsanity.com/paper/1605.06878/) (2016)
- [Model Pruning Based on Quantified Similarity of Feature Maps](https://mlsanity.com/paper/2105.06052/) (2021)
- [Evaluation of Distance Measures for Feature based Image Registration using AlexNet](https://mlsanity.com/paper/1907.12921/) (2019)
- [Siamese Basis Function Networks for Data-efficient Defect Classification in Technical Domains](https://mlsanity.com/paper/2012.01338/) (2020)

## Abstract (arXiv)

Ensembles of Convolutional neural networks have shown remarkable results in learning discriminative semantic features for image classification tasks. Though, the models in the ensemble often concentrate on similar regions in images. This work proposes a novel method that forces a set of base models to learn different features for a classification task. These models are combined in an ensemble to make a collective classification. The key finding is that by forcing the models to concentrate on different features, the classification accuracy is increased. To learn different feature concepts, a so-called feature distance loss is implemented on the feature maps. The experiments on benchmark convolutional neural networks (VGG16, ResNet, AlexNet), popular datasets (Cifar10, Cifar100, miniImageNet, NEU, BSD, TEX), and different training samples (3, 5, 10, 20, 50, 100 per class) show the effectiveness of the proposed feature loss. The proposed method outperforms classical ensemble versions of the base models. The Class Activation Maps explicitly prove the ability to learn different feature concepts. The code is available at: https://github.com/2Obe/Feature-Distance-Loss.git