# OPT-GAN: A Broad-Spectrum Global Optimizer for Black-box Problems by Learning Distribution

arXiv: https://arxiv.org/abs/2102.03888 · HTML: https://mlsanity.com/paper/2102.03888/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 103 references that are in the map.
- First posted on arXiv 7 Feb 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting](https://mlsanity.com/paper/1603.08785/) (2016)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining](https://mlsanity.com/paper/2006.09191/) (2020)
- [Conditioning by adaptive sampling for robust design](https://mlsanity.com/paper/1901.10060/) (2019)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [Gradientless Descent: High-Dimensional Zeroth-Order Optimization](https://mlsanity.com/paper/1911.06317/) (2019)
- [Bayesian Optimization of Risk Measures](https://mlsanity.com/paper/2007.05554/) (2020)
- [Deep Reinforcement Learning based Automatic Exploration for Navigation in Unknown Environment](https://mlsanity.com/paper/2007.11808/) (2020)
- [Design by adaptive sampling](https://mlsanity.com/paper/1810.03714/) (2018)
- [Multi-objective training of Generative Adversarial Networks with multiple discriminators](https://mlsanity.com/paper/1901.08680/) (2019)
- [Pareto GAN: Extending the Representational Power of GANs to Heavy-Tailed Distributions](https://mlsanity.com/paper/2101.09113/) (2021)
- [Evolutionary Multiobjective Optimization Driven by Generative Adversarial Networks (GANs)](https://mlsanity.com/paper/1910.04966/) (2019)
- [High Dimensional Bayesian Optimization Assisted by Principal Component Analysis](https://mlsanity.com/paper/2007.00925/) (2020)
- [Stabilizing Training of Generative Adversarial Nets via Langevin Stein Variational Gradient Descent](https://mlsanity.com/paper/2004.10495/) (2020)
- [Explicit Gradient Learning](https://mlsanity.com/paper/2006.08711/) (2020)
- [AdaDGS: An adaptive black-box optimization method with a nonlocal directional Gaussian smoothing gradient](https://mlsanity.com/paper/2011.02009/) (2020)
- [Enhancing Gaussian Estimation of Distribution Algorithm by Exploiting Evolution Direction with Archive](https://mlsanity.com/paper/1802.08989/) (2018)
- [Global optimization test problems based on random field composition](https://mlsanity.com/paper/1807.05096/) (2018)
- [Improving Evolutionary Strategies with Generative Neural Networks](https://mlsanity.com/paper/1901.11271/) (2019)
- [A Novel Graphic Bending Transformation on Benchmark](https://mlsanity.com/paper/2004.10042/) (2020)

## Built on by (most cited first, 2 of 2)

- [Reinforced In-Context Black-Box Optimization](https://mlsanity.com/paper/2402.17423/) (2024)
- [Learning Distributions via Monte-Carlo Marginalization](https://mlsanity.com/paper/2308.06352/) (2023)

## Abstract (arXiv)

Black-box optimization (BBO) algorithms are concerned with finding the best solutions for problems with missing analytical details. Most classical methods for such problems are based on strong and fixed a priori assumptions, such as Gaussianity. However, the complex real-world problems, especially when the global optimum is desired, could be very far from the a priori assumptions because of their diversities, causing unexpected obstacles. In this study, we propose a generative adversarial net-based broad-spectrum global optimizer (OPT-GAN) which estimates the distribution of optimum gradually, with strategies to balance exploration-exploitation trade-off. It has potential to better adapt to the regularity and structure of diversified landscapes than other methods with fixed prior, e.g., Gaussian assumption or separability. Experiments on diverse BBO benchmarks and high dimensional real world applications exhibit that OPT-GAN outperforms other traditional and neural net-based BBO algorithms.