mlsanityData to 14 Oct 2026

On the Effectiveness of Neural Ensembles for Image Classification with Small Datasets

arXiv 2111.14493 ↗First posted 29 Nov 2021cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep neural networks represent the gold standard for image classification. However, they usually need large amounts of data to reach superior performance. In this work, we focus on image classification problems with a few labeled examples per class and improve data efficiency by using an ensemble of relatively small networks. For the first time, our work broadly studies the existing concept of neural ensembling in domains with small data, through extensive validation using popular datasets and architectures. We compare ensembles of networks to their deeper or wider single competitors given a total fixed computational budget. We show that ensembling relatively shallow networks is a simple yet effective technique that is generally better than current state-of-the-art approaches for learning from small datasets. Finally, we present our interpretation according to which neural ensembles are more sample efficient because they learn simpler functions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 44 references that are in the map. First posted on arXiv 29 Nov 2021.

Builds on

27 of 44 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2016Wide Residual Networkscited by 3,070
  5. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  6. 2018A Survey on Deep Transfer Learningcited by 419
  7. 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
  8. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
  9. 2016On the Expressive Power of Deep Neural Networkscited by 306
  10. 2018Meta-Learning: A Surveycited by 256
  11. 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 189
  12. 2017Generalization in Deep Learningcited by 146
15 more
  1. 2018Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networkscited by 142
  2. 2016Robust Large Margin Deep Neural Networkscited by 118
  3. 2013On the number of response regions of deep feed forward networks with piece-wise linear activationscited by 94
  4. 2019Harnessing the Power of Infinitely Wide Deep Nets on Small-data Taskscited by 87
  5. 2019Scaling description of generalization with number of parameters in deep learningcited by 83
  6. 2019Diversity with Cooperation: Ensemble Methods for Few-Shot Classificationcited by 70
  7. 2020A Close Look at Deep Learning with Small Datacited by 54
  8. 2019Deep Learning on Small Datasets without Pre-Training using Cosine Losscited by 41
  9. 2020Small Data, Big Decisions: Model Selection in the Small-Data Regimecited by 23
  10. 2019Generative Modeling for Small-Data Object Detectioncited by 20
  11. 2018MotherNets: Rapid Deep Ensemble Learningcited by 14
  12. 2019Towards Learning Affine-Invariant Representations via Data-Efficient CNNscited by 10
  13. 2017Coupled Ensembles of Neural Networkscited by 8
  14. 2016Frankenstein: Learning Deep Face Representations using Small Datacited by 7
  15. 2020Deep Ensembles on a Fixed Memory Budget: One Wide Network or Several Thinner Ones?cited by 6

Built on by

Not cited yet.