mlsanityData to 14 Oct 2026

Small Data Explainer -- The impact of small data methods in everyday life

arXiv 2507.11773 ↗First posted 15 Jul 2025cs.CY cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The emergence of breakthrough artificial intelligence (AI) techniques has led to a renewed focus on how small data settings, i.e., settings with limited information, can benefit from such developments. This includes societal issues such as how best to include under-represented groups in data-driven policy and decision making, or the health benefits of assistive technologies. We provide a conceptual overview, clarify the relationship between small data and big data, and identify common themes from exemplary case studies and application areas. Potential solutions are described in a more detailed technical overview of current data analysis and modelling techniques, highlighting contributions from different disciplines, such as knowledge-driven modelling from statistics and data-driven modelling from computer science. By linking application settings, conceptual contributions and specific techniques, we highlight what is already feasible and suggest what an agenda for fully leveraging small data might look like.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 59 of its 177 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

59 of 177 references.

References · 59

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2018Neural Ordinary Differential Equationscited by 3,779
  10. 2016Matching Networks for One Shot Learningcited by 2,255
  11. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  12. 2021Transformers in Vision: A Surveycited by 1,013
38 more
  1. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  2. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  3. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  4. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
  5. 2020A Survey on Visual Transformercited by 654
  6. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  7. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 514
  8. 2021Transformers Can Do Bayesian Inferencecited by 360
  9. 2020Universal Differential Equations for Scientific Machine Learningcited by 353
  10. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  11. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 312
  12. 2020Neurosymbolic AI: The 3rd Wavecited by 255
  13. 2018Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equationscited by 235
  14. 2019Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systemscited by 205
  15. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 169
  16. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  17. 2023mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Videocited by 154
  18. 2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 150
  19. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 145
  20. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  21. 2019Convergence of Edge Computing and Deep Learning: A Comprehensive Surveycited by 120
  22. 2018ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositingcited by 90
  23. 2019A Differentiable Programming System to Bridge Machine Learning and Scientific Computingcited by 83
  24. 2023How Secure is Code Generated by ChatGPT?cited by 81
  25. 2022Large-scale Text-to-Image Generation Models for Visual Artists' Creative Workscited by 61
  26. 2021Improved Transformer for High-Resolution GANscited by 59
  27. 2021Incorporating Domain Knowledge into Deep Neural Networkscited by 56
  28. 2022TTS-GAN: A Transformer-based Time-Series Generative Adversarial Networkcited by 41
  29. 2019Dataset2Vec: Learning Dataset Meta-Featurescited by 38
  30. 2022On the Power of Foundation Modelscited by 37
  31. 2022Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Modelcited by 25
  32. 2020Small Data, Big Decisions: Model Selection in the Small-Data Regimecited by 23
  33. 2022FHIST: A Benchmark for Few-shot Classification of Histological Imagescited by 20
  34. 2020An Overview of Deep Learning Architectures in Few-Shot Learning Domaincited by 16
  35. 2022Visual Semantic Segmentation Based on Few/Zero-Shot Learning: An Overviewcited by 16
  36. 2019Blending Diverse Physical Priors with Neural Networkscited by 11
  37. 2021Machine learning on small size samples: A synthetic knowledge synthesiscited by 11
  38. 2020Learning similarity measures from datacited by 8

50 most cited of 59 All 59 (JSON)

Built on by

Not cited yet.