# Biases in In Silico Evaluation of Molecular Optimization Methods and Bias-Reduced Evaluation Methodology

arXiv: https://arxiv.org/abs/2201.12163 · HTML: https://mlsanity.com/paper/2201.12163/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 28 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [GuacaMol: Benchmarking Models for De Novo Molecular Design](https://mlsanity.com/paper/1811.09621/) (2018)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models](https://mlsanity.com/paper/1811.12823/) (2018)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Learning Deep Generative Models of Graphs](https://mlsanity.com/paper/1803.03324/) (2018)
- [Doubly Robust Policy Evaluation and Optimization](https://mlsanity.com/paper/1503.02834/) (2015)
- [MARS: Markov Molecular Sampling for Multi-objective Drug Discovery](https://mlsanity.com/paper/2103.10432/) (2021)
- [Multi-Objective De Novo Drug Design with Conditional Graph Generative Model](https://mlsanity.com/paper/1801.07299/) (2018)
- [Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning](https://mlsanity.com/paper/2004.12485/) (2020)
- [Molecular Hypergraph Grammar with its Application to Molecular Optimization](https://mlsanity.com/paper/1809.02745/) (2018)
- [Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation](https://mlsanity.com/paper/1910.07186/) (2019)
- [Non-Gaussianity of Stochastic Gradient Noise](https://mlsanity.com/paper/1910.09626/) (2019)
- [Molecular Inverse-Design Platform for Material Industries](https://mlsanity.com/paper/2004.11521/) (2020)

## Built on by (most cited first, 1 of 1)

- [Automated patent extraction powers generative modeling in focused chemical spaces](https://mlsanity.com/paper/2303.08272/) (2023)

## Abstract (arXiv)

We are interested in in silico evaluation methodology for molecular optimization methods. Given a sample of molecules and their properties of our interest, we wish not only to train an agent that can find molecules optimized with respect to the target property but also to evaluate its performance. A common practice is to train a predictor of the target property on the sample and use it for both training and evaluating the agent. We show that this evaluator potentially suffers from two biases; one is due to misspecification of the predictor and the other to reusing the same sample for training and evaluation. We discuss bias reduction methods for each of the biases comprehensively, and empirically investigate their effectiveness.