mlsanityData to 7 Oct 2026

Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimization

arXiv 2304.03995 ↗First posted 8 Apr 2023cs.NE cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

Genetic algorithms constitute a family of black-box optimization algorithms, which take inspiration from the principles of biological evolution. While they provide a general-purpose tool for optimization, their particular instantiations can be heuristic and motivated by loose biological intuition. In this work we explore a fundamentally different approach: Given a sufficiently flexible parametrization of the genetic operators, we discover entirely new genetic algorithms in a data-driven fashion. More specifically, we parametrize selection and mutation rate adaptation as cross- and self-attention modules and use Meta-Black-Box-Optimization to evolve their parameters on a set of diverse optimization tasks. The resulting Learned Genetic Algorithm outperforms state-of-the-art adaptive baseline genetic algorithms and generalizes far beyond its meta-training settings. The learned algorithm can be applied to previously unseen optimization problems, search dimensions & evaluation budgets. We conduct extensive analysis of the discovered operators and provide ablation experiments, which highlight the benefits of flexible module parametrization and the ability to transfer (`plug-in') the learned operators to conventional genetic algorithms.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 27 of its 51 references that are in the map. First posted on arXiv 8 Apr 2023.

Builds on

27 of 51 references.

References · 27

  1. 2016Learning to learn by gradient descent by gradient descentcited by 616
  2. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  3. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  4. 2016Learning to reinforcement learncited by 369
  5. 2018Meta-Gradient Reinforcement Learningcited by 156
  6. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 112
  7. 2017Quality and Diversity Optimization: A Unifying Modular Frameworkcited by 111
  8. 2022Discovered Policy Optimisationcited by 103
  9. 2018Understanding and correcting pathologies in the training of learned optimizerscited by 97
  10. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  11. 2020Discovering Reinforcement Learning Algorithmscited by 75
  12. 2021HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenMLcited by 75
15 more
  1. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  2. 2020Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselvescited by 57
  3. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  4. 2020Meta-Gradient Reinforcement Learning with an Objective Discovered Onlinecited by 52
  5. 2021The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learningcited by 50
  6. 2020Meta Learning Backpropagation And Improving Itcited by 49
  7. 2022Discovering Evolution Strategies via Meta-Black-Box Optimizationcited by 44
  8. 2022evosax: JAX-based Evolution Strategiescited by 39
  9. 2022EvoJAX: Hardware-Accelerated Neuroevolutioncited by 29
  10. 2021Introducing Symmetries to Black Box Meta Reinforcement Learningcited by 25
  11. 2013PEGASUS: A Policy Search Method for Large MDPs and POMDPscited by 21
  12. 2019Meta-Learning for Black-box Optimizationcited by 18
  13. 2021Meta Learning Black-Box Population-Based Optimizerscited by 13
  14. 2022Effective Mutation Rate Adaptation through Group Elite Selectioncited by 6
  15. 2020Learning Not to Learn: Nature versus Nurture in Silicocited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 16; 2025: 6; 2026: 5.

33 papers.

Sort:

By area:

  1. 2024Discovering Preference Optimization Algorithms with and for Large Language Modelscited by 38
  2. 2023MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learningcited by 35
  3. 2024Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimizationcited by 23
  4. 2024Symbol: Generating Flexible Black-Box Optimizers through Symbolic Equation Learningcited by 22
  5. 2024Position: Leverage Foundational Models for Black-Box Optimizationcited by 21
  6. 2024When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challengescited by 20
  7. 2024Discovering Temporally-Aware Reinforcement Learning Algorithmscited by 18
  8. 2024Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learningcited by 17
  9. 2023B2Opt: Learning to Optimize Black-box Optimization with Little Budgetcited by 15
  10. 2024Reinforced In-Context Black-Box Optimizationcited by 14
  11. 2024Pretrained Optimization Model for Zero-Shot Black Box Optimizationcited by 13
  12. 2024Evolution Transformer: In-Context Evolutionary Optimizationcited by 9
21 more
  1. 2025MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimizationcited by 9
  2. 2024RLEMMO: Evolutionary Multimodal Optimization Assisted By Deep Reinforcement Learningcited by 7
  3. 2025Surrogate Learning in Meta-Black-Box Optimization: A Preliminary Studycited by 7
  4. 2023EvoFed: Leveraging Evolutionary Strategies for Communication-Efficient Federated Learningcited by 6
  5. 2023NeuroEvoBench: Benchmarking Evolutionary Optimizers for Deep Learning Applicationscited by 4
  6. 2024Creative Text-to-Audio Generation via Synthesizer Programmingcited by 4
  7. 2024Discovering Minimal Reinforcement Learning Environmentscited by 3
  8. 2025Discovering Quality-Diversity Algorithms via Meta-Black-Box Optimizationcited by 2
  9. 2025Accelerating Vehicle Routing via AI-Initialized Genetic Algorithmscited by 2
  10. 2026Task-free Adaptive Meta Black-box Optimizationcited by 2
  11. 2023Lottery Tickets in Evolutionary Optimization: On Sparse Backpropagation-Free Trainabilitycited by 1
  12. 2024SONATA: Self-adaptive Evolutionary Framework for Hardware-aware Neural Architecture Searchcited by 1
  13. 2025Learning Evolution via Optimization Knowledge Adaptationcited by 1
  14. 2026Detect and Act: Automated Dynamic Optimizer through Meta-Black-Box Optimizationcited by 1
  15. 2023Ever Evolving Evaluator (EV3): Towards Flexible and Reliable Meta-Optimization for Knowledge Distillationcited by 0
  16. 2024Searching Search Spaces: Meta-evolving a Geometric Encoding for Neural Networkscited by 0
  17. 2024An Offline Meta Black-box Optimization Framework for Adaptive Design of Urban Traffic Light Management Systemscited by 0
  18. 2025Discovering new robust local search algorithms with neuro-evolutioncited by 0
  19. 2026NeurGO: Learning to Generate Elite Candidates for Meta-Black-Box Expensive Optimizationcited by 0
  20. 2026Rethinking Learnability in Offline Data-driven Optimizationcited by 0
  21. 2026Hyper Algorithm Design Agent: Evolving Learnable Optimizer from Zerocited by 0