# Unravelling Small Sample Size Problems in the Deep Learning World

arXiv: https://arxiv.org/abs/2008.03522 · HTML: https://mlsanity.com/paper/2008.03522/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2020.

## Built on by (most cited first, 6 of 6)

- [Toward Open-World Electroencephalogram Decoding Via Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2112.06654/) (2021)
- [Detection of financial opportunities in micro-blogging data with a stacked classification system](https://mlsanity.com/paper/2404.07224/) (2024)
- [Acoustic-Linguistic Features for Modeling Neurological Task Score in Alzheimer's](https://mlsanity.com/paper/2209.06085/) (2022)
- [Gauge-optimal approximate learning for small data classification problems](https://mlsanity.com/paper/2310.19066/) (2023)
- [A novel RNA pseudouridine site prediction model using Utility Kernel and data-driven parameters](https://mlsanity.com/paper/2311.16132/) (2023)
- [Exploring the Impact of Dataset Statistical Effect Size on Model Performance and Data Sample Size Sufficiency](https://mlsanity.com/paper/2501.02673/) (2025)

## Abstract (arXiv)

The growth and success of deep learning approaches can be attributed to two major factors: availability of hardware resources and availability of large number of training samples. For problems with large training databases, deep learning models have achieved superlative performances. However, there are a lot of textitsmall sample size or S3 problems for which it is not feasible to collect large training databases. It has been observed that deep learning models do not generalize well on S3 problems and specialized solutions are required. In this paper, we first present a review of deep learning algorithms for small sample size problems in which the algorithms are segregated according to the space in which they operate, i.e. input space, model space, and feature space. Secondly, we present Dynamic Attention Pooling approach which focuses on extracting global information from the most discriminative sub-part of the feature map. The performance of the proposed dynamic attention pooling is analyzed with state-of-the-art ResNet model on relatively small publicly available datasets such as SVHN, C10, C100, and TinyImageNet.