The Lean Data Scientist: Recent Advances towards Overcoming the Data Bottleneck
Not on the map. Why?
Abstract (arXiv)
Machine learning (ML) is revolutionizing the world, affecting almost every field of science and industry. Recent algorithms (in particular, deep networks) are increasingly data-hungry, requiring large datasets for training. Thus, the dominant paradigm in ML today involves constructing large, task-specific datasets. However, obtaining quality datasets of such magnitude proves to be a difficult challenge. A variety of methods have been proposed to address this data bottleneck problem, but they are scattered across different areas, and it is hard for a practitioner to keep up with the latest developments. In this work, we propose a taxonomy of these methods. Our goal is twofold: (1) We wish to raise the community's awareness of the methods that already exist and encourage more efficient use of resources, and (2) we hope that such a taxonomy will contribute to our understanding of the problem, inspiring novel ideas and strategies to replace current annotation-heavy approaches.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 58 of its 136 references that are in the map. First posted on arXiv 15 Nov 2022.
Builds on
58 of 136 references.
References · 58
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
- 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
- 2016Pixel Recurrent Neural Networkscited by 814
- 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
- 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
38 more
- 2018Annotation Artifacts in Natural Language Inference Datacited by 656
- 2019Green AIcited by 620
- 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
- 2020A Survey of Deep Active Learningcited by 547
- 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 459
- 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
- 2017The Effectiveness of Data Augmentation in Image Classification using Deep Learningcited by 368
- 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
- 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
- 2020CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognitioncited by 320
- 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
- 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
- 2017Data Augmentation Generative Adversarial Networkscited by 278
- 2017Meta Networkscited by 268
- 2017Automated Curriculum Learning for Neural Networkscited by 251
- 2015Semi-Supervised Learning with Ladder Networkscited by 246
- 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
- 2018Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoderscited by 206
- 2017Teacher-Student Curriculum Learningcited by 197
- 2019Not Enough Data? Deep Learning to the Rescue!cited by 174
- 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
- 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 131
- 2019Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentationcited by 126
- 2016Synthesized Classifiers for Zero-Shot Learningcited by 113
- 2018A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspectivecited by 111
- 2018GAN Augmentation: Augmenting Training Data using Generative Adversarial Networkscited by 107
- 2014Learning to Executecited by 105
- 2017Learning to Compose Domain-Specific Transformations for Data Augmentationcited by 91
- 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 79
- 2017Generative Adversarial Active Learningcited by 69
- 2019Bayesian Generative Active Deep Learningcited by 57
- 2019Let's Agree to Agree: Neural Networks Share Classification Order on Real Datasetscited by 55
- 2016Less is more: zero-shot learning from online textual documents with noise suppressioncited by 40
- 2017Improving Deep Learning using Generic Data Augmentationcited by 36
- 2019Baby steps towards few-shot learning with multiple semanticscited by 29
- 2019PEPSI++: Fast and Lightweight Network for Image Inpaintingcited by 24
- 2018Small Sample Learning in Big Data Eracited by 18
- 2019Reverse-Engineering Satire, or "Paper on Computational Humor Accepted Despite Making Serious Advances"cited by 15
50 most cited of 58 All 58 (JSON)
Built on by
Citing papers per year: 2023: 1.
1 paper.