mlsanityData to 7 Oct 2026

Reinforced In-Context Black-Box Optimization

arXiv 2402.17423 ↗First posted 27 Feb 2024cs.LG cs.AI cs.NEcited by 14

Not on the map. Why?

Abstract (arXiv)

Black-Box Optimization (BBO) has found successful applications in many fields of science and engineering. Recently, there has been a growing interest in meta-learning particular components of BBO algorithms to speed up optimization and get rid of tedious hand-crafted heuristics. As an extension, learning the entire algorithm from data requires the least labor from experts and can provide the most flexibility. In this paper, we propose RIBBO, a method to reinforce-learn a BBO algorithm from offline data in an end-to-end fashion. RIBBO employs expressive sequence models to learn the optimization histories produced by multiple behavior algorithms and tasks, leveraging the in-context learning ability of large models to extract task information and make decisions accordingly. Central to our method is to augment the optimization histories with textitregret-to-go tokens, which are designed to represent the performance of an algorithm based on cumulative regret over the future part of the histories. The integration of regret-to-go tokens enables RIBBO to automatically generate sequences of query points that satisfy the user-desired regret, which is verified by its universally good empirical performance on diverse problems, including BBO benchmark functions, hyper-parameter optimization and robot control problems.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 54 of its 78 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

54 of 78 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  6. 2021Transformers in Vision: A Surveycited by 1,009
  7. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  8. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  9. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  10. 2018A Tutorial on Bayesian Optimizationcited by 734
  11. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  12. 2022Transformers in Time Series: A Surveycited by 489
38 more
  1. 2018Conditional Neural Processescited by 421
  2. 2021A Survey of Transformerscited by 399
  3. 2016The CMA Evolution Strategy: A Tutorialcited by 364
  4. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  5. 2021Transformers Can Do Bayesian Inferencecited by 338
  6. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 220
  7. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 186
  8. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  9. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 126
  10. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  11. 2018Maximizing acquisition functions for Bayesian optimizationcited by 113
  12. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  13. 2015Bayesian optimization for materials designcited by 78
  14. 2021HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenMLcited by 75
  15. 2023How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representationscited by 72
  16. 2019Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimizationcited by 70
  17. 2021Few-Shot Bayesian Optimization with Deep Kernel Surrogatescited by 70
  18. 2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 70
  19. 2023Towards Omni-generalizable Neural Methods for Vehicle Routing Problemscited by 68
  20. 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
  21. 2019OpenML-Python: an extensible Python API for OpenMLcited by 62
  22. 2019Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learningcited by 50
  23. 2022Discovering Evolution Strategies via Meta-Black-Box Optimizationcited by 44
  24. 2022Automated Dynamic Algorithm Configurationcited by 37
  25. 2022Generative Pretraining for Black-Box Optimizationcited by 36
  26. 2022Multi-agent Dynamic Algorithm Configurationcited by 36
  27. 2024OmniPred: Language Models as Universal Regressorscited by 34
  28. 2018A survey on policy search algorithms for learning robot controllers in a handful of trialscited by 33
  29. 2023Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimizationcited by 33
  30. 2023End-to-End Meta-Bayesian Optimisation with Transformer Neural Processescited by 31
  31. 2023Transfer Learning for Bayesian Optimization: A Surveycited by 27
  32. 2022Thinking inside the box: A tutorial on grey-box Bayesian optimizationcited by 26
  33. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioningcited by 24
  34. 2023Macro Placement by Wire-Mask-Guided Black-Box Optimizationcited by 23
  35. 2021Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimizationcited by 22
  36. 2023ExPT: Synthetic Pretraining for Few-Shot Experimental Designcited by 22
  37. 2016Warm Starting Bayesian Optimizationcited by 21
  38. 2019Meta-Learning for Black-box Optimizationcited by 18

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 5.

14 papers.

Sort:

By area:

  1. 2024Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimizationcited by 23
  2. 2024FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearchcited by 22
  3. 2024Neural Exploratory Landscape Analysis for Meta-Black-Box-Optimizationcited by 10
  4. 2025PABBO: Preferential Amortized Black-Box Optimizationcited by 7
  5. 2024EARL-BO: Reinforcement Learning for Multi-Step Lookahead, High-Dimensional Bayesian Optimizationcited by 6
  6. 2025Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddingscited by 6
  7. 2025In-Context Multi-Objective Optimizationcited by 5
  8. 2026From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priorscited by 2
  9. 2026Efficient Adaptive Data Acquisition via Pretrained Belief Representationscited by 2
  10. 2025ZeroShotOpt: Towards Zero-Shot Pretrained Models for Efficient Black-Box Optimizationcited by 1
  11. 2025SemanticOpt: Towards LLM-Based Semantic Black-Box Optimizationcited by 1
  12. 2026NeurGO: Learning to Generate Elite Candidates for Meta-Black-Box Expensive Optimizationcited by 0
2 more
  1. 2026Rethinking Learnability in Offline Data-driven Optimizationcited by 0
  2. 2026Frontier LLMs are effective batch optimizers: Assessing reasoning models in continuous and discrete settingscited by 0