Guided Evolution with Binary Discriminators for ML Program Search
Not on the map. Why?
Abstract (arXiv)
How to automatically design better machine learning programs is an open problem within AutoML. While evolution has been a popular tool to search for better ML programs, using learning itself to guide the search has been less successful and less understood on harder problems but has the promise to dramatically increase the speed and final performance of the optimization process. We propose guiding evolution with a binary discriminator, trained online to distinguish which program is better given a pair of programs. The discriminator selects better programs without having to perform a costly evaluation and thus speed up the convergence of evolution. Our method can encode a wide variety of ML components including symbolic optimizers, neural architectures, RL loss functions, and symbolic regression equations with the same directed acyclic graph representation. By combining this representation with modern GNNs and an adaptive mutation strategy, we demonstrate our method can speed up evolution across a set of diverse problems including a 3.7x speedup on the symbolic search for ML optimizers and a 4x speedup for RL loss functions.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 8 Feb 2024.
Builds on
31 of 41 references.
References · 31
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2018How Powerful are Graph Neural Networks?cited by 4,355
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
- 2018DARTS: Differentiable Architecture Searchcited by 1,782
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
- 2019Strategies for Pre-training Graph Neural Networkscited by 964
- 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
- 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
- 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
- 2017Progressive Neural Architecture Searchcited by 483
- 2019AutoML: A Survey of the State-of-the-Artcited by 388
19 more
- 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
- 2019Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradientscited by 294
- 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
- 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
- 2022Structure-Aware Transformer for Graph Representation Learningcited by 191
- 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
- 2020AutoML-Zero: Evolving Machine Learning Algorithms From Scratchcited by 137
- 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 129
- 2019Neural Predictor for Neural Architecture Searchcited by 116
- 2020BRP-NAS: Prediction-based NAS using GCNscited by 109
- 2021Symbolic Regression via Neural-Guided Genetic Programming Population Seedingcited by 97
- 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 88
- 2017Neural Optimizer Search with Reinforcement Learningcited by 82
- 2021Evolving Reinforcement Learning Algorithmscited by 48
- 2017Simple And Efficient Architecture Search for Convolutional Neural Networkscited by 40
- 2019ReNAS:Relativistic Evaluation of Neural Architecture Searchcited by 35
- 2021Contrastive Neural Architecture Search with Neural Architecture Comparatorscited by 33
- 2018Neural Program Synthesis with Priority Queue Trainingcited by 24
- 2022PRE-NAS: Predictor-assisted Evolutionary Neural Architecture Searchcited by 8
Built on by
Not cited yet.