mlsanityData to 14 Oct 2026

Adaptive Partitioning Schemes for Optimistic Optimization

arXiv 2610.04039 ↗First posted 2 Oct 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Applications such as engineering design often require us to optimize a black-box function, i.e., a system whose inner processing is not analytically known and whose gradients are not available. Practitioners often have a fixed budget for the number of function evaluations and the performance of an optimization algorithm is measured by its simple regret. In this paper, we study the class of "Optimistic Optimization" algorithms for black-box optimization that use a partitioning scheme for the domain. We develop algorithms that learn a good partitioning scheme and use flexible surrogate models such as neural networks in the optimization procedure. For multi-index functions on an m-dimensional subspace within d dimensions, our algorithm attains tildeO(n-β / d) regret, where β = 1 + fracd-m2m-1, as opposed to tildeO(n-1/d) for SequOOL, a state-of-the-art optimistic optimization algorithm. We use our approach to improve the quality of Activation-aware Weight Quantization (AWQ) of the OPT-1.3B model, achieving ~10% improvement in performance relative to the best possible unquantized model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 34 references that are in the map. First posted on arXiv 2 Oct 2026.

Builds on

20 of 34 references.

References · 20

  1. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  5. 2016The CMA Evolution Strategy: A Tutorialcited by 377
  6. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
  7. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 182
  8. 2020Spectral Methods for Data Science: A Statistical Perspectivecited by 112
  9. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 92
  10. 2020Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Searchcited by 89
  11. 2022Neural Networks Efficiently Learn Low-Dimensional Representations with SGDcited by 63
  12. 2016CMA-ES for Hyperparameter Optimization of Deep Neural Networkscited by 48
8 more
  1. 2021Near-Minimax Optimal Estimation With Shallow ReLU Neural Networkscited by 41
  2. 2020A Domain-Shrinking based Bayesian Optimization Algorithm with Order-Optimal Regret Performancecited by 39
  3. 2018A simple parameter-free and adaptive approach to optimization under a minimal local smoothness assumptioncited by 23
  4. 2017Gaussian Process bandits with adaptive discretizationcited by 21
  5. 2020Warm Starting CMA-ES for Hyperparameter Optimizationcited by 19
  6. 2020Chernoff Sampling for Active Testing and Extension to Active Regressioncited by 13
  7. 2026Stochastic simultaneous optimistic optimizationcited by 8
  8. 2026Black-box optimization of noisy functions with unknown smoothnesscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization
  2. 2022LAMBDA: Covering the Solution Set of Black-Box Inequality by Search Space Quantization
  3. 2026Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputs
  4. 2022Combinatorial optimization for low bit-width neural networks
  5. 2023Diffusion Models for Black-Box Optimization
  6. 2025High-Dimensional Search, Low-Dimensional Solution: Decoupling Optimization from Representation
  7. 2026In-Context Batch Black-box Optimization via Flow Matching
  8. 2024Robust Guided Diffusion for Offline Black-Box Optimization