mlsanityData to 7 Oct 2026

Winning solutions and post-challenge analyses of the ChaLearn AutoDL challenge 2019

arXiv 2201.03801 ↗First posted 11 Jan 2022cs.LG cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

This paper reports the results and post-challenge analyses of ChaLearn's AutoDL challenge series, which helped sorting out a profusion of AutoML solutions for Deep Learning (DL) that had been introduced in a variety of settings, but lacked fair comparisons. All input data modalities (time series, images, videos, text, tabular) were formatted as tensors and all tasks were multi-label classification problems. Code submissions were executed on hidden tasks, with limited time and computational resources, pushing solutions that get results quickly. In this setting, DL methods dominated, though popular Neural Architecture Search (NAS) was impractical. Solutions relied on fine-tuned pre-trained networks, with architectures matching data modality. Post-challenge tests did not reveal improvements beyond the imposed time limit. While no component is particularly original or novel, a high level modular organization emerged featuring a "meta-learner", "data ingestor", "model selector", "model/learner", and "evaluator". This modularity enabled ablation studies, which revealed the importance of (off-platform) meta-learning, ensembling, and efficient data management. Experiments on heterogeneous module combinations further confirm the (local) optimality of the winning solutions. Our challenge legacy includes an ever-lasting benchmark (http://autodl.chalearn.org), the open-sourced code of the winners, and a free "AutoDL self-service".

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 30 of its 81 references that are in the map. First posted on arXiv 11 Jan 2022.

Builds on

30 of 81 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2017Squeeze-and-Excitation Networkscited by 4,763
  9. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  10. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  11. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  12. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
18 more
  1. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  2. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  3. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  4. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  5. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  6. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  7. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  8. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  9. 2017Large-Scale Evolution of Image Classifierscited by 348
  10. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  11. 2018Meta-Learning: A Surveycited by 254
  12. 2019Fast AutoAugmentcited by 243
  13. 2020AutoML-Zero: Evolving Machine Learning Algorithms From Scratchcited by 137
  14. 2019NAS evaluation is frustratingly hardcited by 69
  15. 2016AdaNet: Adaptive Structural Learning of Artificial Neural Networkscited by 54
  16. 2019Utterance-level Aggregation For Speaker Recognition In The Wildcited by 49
  17. 2017Probabilistic Matrix Factorization for Automated Machine Learningcited by 31
  18. 2017DeepArchitect: Automatically Designing and Training Deep Architecturescited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 4; 2024: 3; 2025: 0; 2026: 1.

16 papers.

Sort:

By area:

  1. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 222
  2. 2022NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxiescited by 36
  3. 2023Meta-Album: Multi-domain Meta-Dataset for Few-Shot Image Classificationcited by 29
  4. 2022Zero-Shot AutoML with Pretrained Modelscited by 13
  5. 2022Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classificationcited by 10
  6. 2022Asynchronous Decentralized Bayesian Optimization for Large Scale Hyperparameter Optimizationcited by 6
  7. 2022Task Selection for AutoML System Evaluationcited by 6
  8. 2022Advances in MetaDL: AAAI 2021 challenge and workshopcited by 4
  9. 2022NeurIPS'22 Cross-Domain MetaDL competition: Design and baseline resultscited by 3
  10. 2024FAIR Universe HiggsML Uncertainty Dataset and Competitioncited by 2
  11. 2026FAIR Universe Weak Lensing ML Uncertainty Challenge: Handling Uncertainties and Distribution Shifts for Precision Cosmologycited by 2
  12. 2022Meta-learning from Learning Curves Challenge: Lessons learned from the First Round and Design of the Second Roundcited by 1
4 more
  1. 2023Variational Boosted Soft Treescited by 0
  2. 2023Machine learning's own Industrial Revolutioncited by 0
  3. 2023Academic competitionscited by 0
  4. 2024Challenge design roadmapcited by 0