# Offline Model-Based Optimization via Normalized Maximum Likelihood Estimation

arXiv: https://arxiv.org/abs/2102.07970 · HTML: https://mlsanity.com/paper/2102.07970/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 39 references that are in the map.
- First posted on arXiv 16 Feb 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [MOPO: Model-based Offline Policy Optimization](https://mlsanity.com/paper/2005.13239/) (2020)
- [MOReL : Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2005.05951/) (2020)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Deep Reinforcement Learning for De-Novo Drug Design](https://mlsanity.com/paper/1711.10907/) (2017)
- [Conditioning by adaptive sampling for robust design](https://mlsanity.com/paper/1901.10060/) (2019)
- [Model Inversion Networks for Model-Based Optimization](https://mlsanity.com/paper/1912.13464/) (2019)
- [Autofocused oracles for model-based design](https://mlsanity.com/paper/2006.08052/) (2020)
- [Generating and designing DNA with deep generative models](https://mlsanity.com/paper/1712.06148/) (2017)
- [Deep pNML: Predictive Normalized Maximum Likelihood for Deep Neural Networks](https://mlsanity.com/paper/1904.12286/) (2019)
- [Universal Supervised Learning for Individual Data](https://mlsanity.com/paper/1812.09520/) (2018)

## Built on by (most cited first, 30 of 38)

- [Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization](https://mlsanity.com/paper/2202.08450/) (2022)
- [Conservative Objective Models for Effective Offline Model-Based Optimization](https://mlsanity.com/paper/2107.06882/) (2021)
- [An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization](https://mlsanity.com/paper/2404.07771/) (2024)
- [RoMA: Robust Model Adaptation for Offline Model-based Optimization](https://mlsanity.com/paper/2110.14188/) (2021)
- [Generative Pretraining for Black-Box Optimization](https://mlsanity.com/paper/2206.10786/) (2022)
- [Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints](https://mlsanity.com/paper/2402.18012/) (2024)
- [Data-Driven Offline Decision-Making via Invariant Representation Learning](https://mlsanity.com/paper/2211.11349/) (2022)
- [Data-Driven Offline Optimization For Architecting Hardware Accelerators](https://mlsanity.com/paper/2110.11346/) (2021)
- [Diffusion Model for Data-Driven Black-Box Optimization](https://mlsanity.com/paper/2403.13219/) (2024)
- [Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection](https://mlsanity.com/paper/2110.09246/) (2021)
- [Bootstrapped Training of Score-Conditioned Generator for Offline Design of Biological Sequences](https://mlsanity.com/paper/2306.03111/) (2023)
- [ExPT: Synthetic Pretraining for Few-Shot Experimental Design](https://mlsanity.com/paper/2310.19961/) (2023)
- [Towards Deeper Deep Reinforcement Learning with Spectral Normalization](https://mlsanity.com/paper/2106.01151/) (2021)
- [Offline Model-Based Optimization via Policy-Guided Gradient Search](https://mlsanity.com/paper/2405.05349/) (2024)
- [ParetoFlow: Guided Flows in Multi-Objective Optimization](https://mlsanity.com/paper/2412.03718/) (2024)
- [Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization](https://mlsanity.com/paper/2407.01624/) (2024)
- [Offline Model-Based Optimization: Comprehensive Review](https://mlsanity.com/paper/2503.17286/) (2025)
- [Offline Model-Based Optimization by Learning to Rank](https://mlsanity.com/paper/2410.11502/) (2024)
- [Design Editing for Offline Model-based Optimization](https://mlsanity.com/paper/2405.13964/) (2024)
- [Automated Crossword Solving](https://mlsanity.com/paper/2205.09665/) (2022)
- [Design from Policies: Conservative Test-Time Adaptation for Offline Policy Optimization](https://mlsanity.com/paper/2306.14479/) (2023)
- ["Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihood](https://mlsanity.com/paper/2405.16730/) (2024)
- [Robust Guided Diffusion for Offline Black-Box Optimization](https://mlsanity.com/paper/2410.00983/) (2024)
- [Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings](https://mlsanity.com/paper/2506.07109/) (2025)
- [Amortized Synthesis of Constrained Configurations Using a Differentiable Surrogate](https://mlsanity.com/paper/2106.09019/) (2021)
- [Boosting Offline Optimizers with Surrogate Sensitivity](https://mlsanity.com/paper/2503.04181/) (2025)
- [Model-based Offline Reinforcement Learning with Local Misspecification](https://mlsanity.com/paper/2301.11426/) (2023)
- [Incorporating Surrogate Gradient Norm to Improve Offline Optimization Techniques](https://mlsanity.com/paper/2503.04242/) (2025)
- [ROOT: Rethinking Offline Optimization as Distributional Translation via Probabilistic Bridge](https://mlsanity.com/paper/2509.16300/) (2025)
- [Conservative objective models are a special kind of contrastive divergence-based energy model](https://mlsanity.com/paper/2304.03866/) (2023)

## Abstract (arXiv)

In this work we consider data-driven optimization problems where one must maximize a function given only queries at a fixed set of points. This problem setting emerges in many domains where function evaluation is a complex and expensive process, such as in the design of materials, vehicles, or neural network architectures. Because the available data typically only covers a small manifold of the possible space of inputs, a principal challenge is to be able to construct algorithms that can reason about uncertainty and out-of-distribution values, since a naive optimizer can easily exploit an estimated model to return adversarial inputs. We propose to tackle this problem by leveraging the normalized maximum-likelihood (NML) estimator, which provides a principled approach to handling uncertainty and out-of-distribution inputs. While in the standard formulation NML is intractable, we propose a tractable approximation that allows us to scale our method to high-capacity neural network models. We demonstrate that our method can effectively optimize high-dimensional design problems in a variety of disciplines such as chemistry, biology, and materials engineering.