mlsanityData to 14 Oct 2026

How to Do Machine Learning with Small Data? -- A Review from an Industrial Perspective

arXiv 2311.07126 ↗First posted 13 Nov 2023cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence experienced a technological breakthrough in science, industry, and everyday life in the recent few decades. The advancements can be credited to the ever-increasing availability and miniaturization of computational resources that resulted in exponential data growth. However, because of the insufficient amount of data in some cases, employing machine learning in solving complex tasks is not straightforward or even possible. As a result, machine learning with small data experiences rising importance in data science and application in several fields. The authors focus on interpreting the general term of "small data" and their engineering and industrial application role. They give a brief overview of the most important industrial applications of machine learning and small data. Small data is defined in terms of various characteristics compared to big data, and a machine learning formalism was introduced. Five critical challenges of machine learning with small data in industrial applications are presented: unlabeled data, imbalanced data, missing data, insufficient data, and rare events. Based on those definitions, an overview of the considerations in domain representation and data acquisition is given along with a taxonomy of machine learning approaches in the context of small data.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 70 of its 282 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

70 of 282 references.

References · 70

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  5. 2013Auto-Encoding Variational Bayescited by 4,053
  6. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  7. 2016Improved Techniques for Training GANscited by 2,970
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  9. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  10. 2016Matching Networks for One Shot Learningcited by 2,255
  11. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  12. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
38 more
  1. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  2. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  3. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  4. 2016End to End Learning for Self-Driving Carscited by 1,009
  5. 2019An Introduction to Variational Autoencoderscited by 1,000
  6. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  7. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  8. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  9. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  10. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
  11. 2017Wasserstein GANcited by 639
  12. 2017Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discoverycited by 566
  13. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  14. 2013Generating Sequences With Recurrent Neural Networkscited by 554
  15. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 538
  16. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 513
  17. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
  18. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 463
  19. 2016Tutorial on Variational Autoencoderscited by 409
  20. 2018GAIN: Missing Data Imputation using Generative Adversarial Netscited by 352
  21. 2015LSTM: A Search Space Odysseycited by 316
  22. 2014Caffe: Convolutional Architecture for Fast Feature Embeddingcited by 288
  23. 2016Learning from Simulated and Unsupervised Images through Adversarial Trainingcited by 288
  24. 2018Meta-Learning: A Surveycited by 256
  25. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  26. 2017One-Shot Imitation Learningcited by 233
  27. 2015A Critical Review of Recurrent Neural Networks for Sequence Learningcited by 224
  28. 2018Learning from positive and unlabeled data: a surveycited by 219
  29. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 191
  30. 2018Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flowscited by 150
  31. 2018Deep Fluids: A Generative Network for Parameterized Fluid Simulationscited by 121
  32. 2016Low Data Drug Discovery with One-shot Learningcited by 91
  33. 2017Estimating Missing Data in Temporal Data Streams Using Multi-directional Recurrent Neural Networkscited by 86
  34. 2017Comparative Study of CNN and RNN for Natural Language Processingcited by 82
  35. 2016Modeling Missing Data in Clinical Time Series with RNNscited by 80
  36. 2017DeLiGAN : Generative Adversarial Networks for Diverse and Limited Datacited by 77
  37. 2017Perceptual Generative Adversarial Networks for Small Object Detectioncited by 76
  38. 2014Toward Supervised Anomaly Detectioncited by 73

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Mind the (Data) Gap: Evaluating Vision Systems in Small Data Applicationscited by 3
  2. 2025Sample-Efficient Bayesian Transfer Learning for Online Machine Parameter Optimizationcited by 0
  3. 2026Autonomous End-to-End SOH Prediction Services for Battery Systems via Temporal-Contrastive Representation Learningcited by 0