# On Diversity in Discriminative Neural Networks

arXiv: https://arxiv.org/abs/2407.12599 · HTML: https://mlsanity.com/paper/2407.12599/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 11 of 11)

- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [Semi-Supervised Learning with Ladder Networks](https://mlsanity.com/paper/1507.02672/) (2015)
- [Dash: Semi-Supervised Learning with Dynamic Thresholding](https://mlsanity.com/paper/2109.00650/) (2021)
- [BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active Annotation](https://mlsanity.com/paper/2203.02533/) (2022)
- [Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks](https://mlsanity.com/paper/2107.11400/) (2021)
- [No Routing Needed Between Capsules](https://mlsanity.com/paper/2001.09136/) (2020)
- [Sparsity through evolutionary pruning prevents neuronal networks from overfitting](https://mlsanity.com/paper/1911.10988/) (2019)

## Abstract (arXiv)

Diversity is a concept of prime importance in almost all disciplines based on information processing. In telecommunications, for example, spatial, temporal, and frequency diversity, as well as redundant coding, are fundamental concepts that have enabled the design of extremely efficient systems. In machine learning, in particular with neural networks, diversity is not always a concept that is emphasized or at least clearly identified. This paper proposes a neural network architecture that builds upon various diversity principles, some of them already known, others more original. Our architecture obtains remarkable results, with a record self-supervised learning accuracy of 99. 57% in MNIST, and a top tier promising semi-supervised learning accuracy of 94.21% in CIFAR-10 using only 25 labels per class.