mlsanityData to 13 Oct 2026

Sharpness-Aware Black-Box Optimization

arXiv 2410.12457 ↗First posted 16 Oct 2024cs.LG cs.AIcited by 2

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Black-box optimization algorithms have been widely used in various machine learning problems, including reinforcement learning and prompt fine-tuning. However, directly optimizing the training loss value, as commonly done in existing black-box optimization methods, could lead to suboptimal model quality and generalization performance. To address those problems in black-box optimization, we propose a novel Sharpness-Aware Black-box Optimization (SABO) algorithm, which applies a sharpness-aware minimization strategy to improve the model generalization. Specifically, the proposed SABO method first reparameterizes the objective function by its expectation over a Gaussian distribution. Then it iteratively updates the parameterized distribution by approximated stochastic gradients of the maximum objective value within a small neighborhood around the current solution in the Gaussian distribution space. Theoretically, we prove the convergence rate and generalization bound of the proposed SABO algorithm. Empirically, extensive experiments on the black-box prompt fine-tuning tasks demonstrate the effectiveness of the proposed SABO method in improving model generalization performance.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 40 of its 59 references that are in the map. First posted on arXiv 16 Oct 2024.

Builds on

40 of 59 references.

References · 40

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,880
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,328
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,870
  5. 2021Convex optimizationcited by 2,488
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,319
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,841
  8. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,411
  9. 2017Exploring Generalization in Deep Learningcited by 596
  10. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 562
  11. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 530
  12. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
28 more
  1. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 363
  2. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  3. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 256
  4. 2019Efficient Decision-based Black-box Adversarial Attacks on Face Recognitioncited by 174
  5. 2022Surrogate Gap Minimization Improves Sharpness-Aware Trainingcited by 172
  6. 2022Towards Efficient and Scalable Sharpness-Aware Minimizationcited by 129
  7. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  8. 2021Sharpness-Aware Minimization Improves Language Model Generalizationcited by 108
  9. 2017Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agentscited by 107
  10. 2020Relative Flatness and Generalizationcited by 88
  11. 2021Conservative Objective Models for Effective Offline Model-Based Optimizationcited by 88
  12. 2022Fisher SAM: Information Geometry and Sharpness Aware Minimisationcited by 84
  13. 2019Model Inversion Networks for Model-Based Optimizationcited by 83
  14. 2018Adversarially Robust Optimization with Gaussian Processescited by 61
  15. 2022BBTv2: Towards a Gradient-Free Future with Large Language Modelscited by 53
  16. 2020Autofocused oracles for model-based designcited by 50
  17. 2023An Adaptive Policy to Employ Sharpness-Aware Minimizationcited by 43
  18. 2022SAM as an Optimal Relaxation of Bayescited by 41
  19. 2019The intriguing role of module criticality in the generalization of deep networkscited by 40
  20. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 36
  21. 2022Bidirectional Learning for Offline Infinite-width Model-based Optimizationcited by 35
  22. 2022Data-Driven Offline Decision-Making via Invariant Representation Learningcited by 31
  23. 2022Local Bayesian optimization via maximizing probability of descentcited by 30
  24. 2022Generalizing Gaussian Smoothing for Random Searchcited by 15
  25. 2022Gradual Domain Adaptation via Normalizing Flowscited by 11
  26. 2022Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learningcited by 11
  27. 2022MMES: Mixture Model based Evolution Strategy for Large-Scale Optimizationcited by 9
  28. 2019Mirror Natural Evolution Strategiescited by 2

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Zeroth-Order Optimization Finds Flat Minimacited by 6
  2. 2025Zeroth-Order Sharpness-Aware Learning with Exponential Tiltingcited by 0