# Lo-Hi: Practical ML Drug Discovery Benchmark

arXiv: https://arxiv.org/abs/2310.06399 · HTML: https://mlsanity.com/paper/2310.06399/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 88 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 23 of 23)

- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [GuacaMol: Benchmarking Models for De Novo Molecular Design](https://mlsanity.com/paper/1811.09621/) (2018)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization](https://mlsanity.com/paper/2206.12411/) (2022)
- [GOOD: A Graph Out-of-Distribution Benchmark](https://mlsanity.com/paper/2206.08452/) (2022)
- [On the State of the Art of Evaluation in Neural Language Models](https://mlsanity.com/paper/1707.05589/) (2017)
- [Exploring Chemical Space with Score-based Out-of-distribution Generation](https://mlsanity.com/paper/2206.07632/) (2022)
- [SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery](https://mlsanity.com/paper/1911.04738/) (2019)
- [Benchmarking Graphormer on Large-Scale Molecular Modeling Datasets](https://mlsanity.com/paper/2203.04810/) (2022)
- [DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations](https://mlsanity.com/paper/2201.09637/) (2022)
- [MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimization](https://mlsanity.com/paper/2010.02318/) (2020)
- [How Much Space Has Been Explored? Measuring the Chemical Space Covered by Databases and Machine-Generated Molecules](https://mlsanity.com/paper/2112.12542/) (2021)
- [Learning Subpocket Prototypes for Generalizable Structure-based Drug Design](https://mlsanity.com/paper/2305.13997/) (2023)
- [Path-Augmented Graph Transformer Network](https://mlsanity.com/paper/1905.12712/) (2019)
- [Bayesian semi-supervised learning for uncertainty-calibrated prediction of molecular properties and active learning](https://mlsanity.com/paper/1902.00925/) (2019)
- [Scaffold-constrained molecular generation](https://mlsanity.com/paper/2009.07778/) (2020)
- [Edge Attention-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1802.04944/) (2018)
- [Activity Cliff Prediction: Dataset and Benchmark](https://mlsanity.com/paper/2302.07541/) (2023)
- [Three-Dimensionally Embedded Graph Convolutional Network (3DGCN) for Molecule Interpretation](https://mlsanity.com/paper/1811.09794/) (2018)

## Built on by (most cited first, 10 of 10)

- [Closed-loop Auto Research for Molecular Property Prediction: Discovering and Certifying Generalizable Improvements](https://mlsanity.com/paper/2606.22731/) (2026)
- [Synergistic Benefits of Joint Molecule Generation and Property Prediction](https://mlsanity.com/paper/2504.16559/) (2025)
- [Benchmarking drug-drug interaction prediction methods: a perspective of distribution changes](https://mlsanity.com/paper/2410.18583/) (2024)
- [Evaluation Framework for AI-driven Molecular Design of Multi-target Drugs: Brain Diseases as a Case Study](https://mlsanity.com/paper/2408.10482/) (2024)
- [MolSafeEval: A Benchmark for Uncovering Safety Risks in AI-Generated Molecules](https://mlsanity.com/paper/2607.00464/) (2026)
- [Bioptic B1: A Target-Agnostic Potency-Based Small Molecules Search Engine](https://mlsanity.com/paper/2406.14572/) (2024)
- [Decomposing the Generalization Gap in PROTAC Activity Prediction: Variance Attribution and the Inter-Laboratory Ceiling](https://mlsanity.com/paper/2605.11764/) (2026)
- [Refnd: Preventing Data Leakage in Relational Datasets](https://mlsanity.com/paper/2607.19376/) (2026)
- [Rigorous Evaluation of Large Language Models for Malaria Drug Discovery: Trade-offs in Performance, Scale, and Resource Utility](https://mlsanity.com/paper/2608.20418/) (2026)
- [Molecular Property Prediction under Structural Shift with Tabular Foundation Models](https://mlsanity.com/paper/2609.38744/) (2026)

## Abstract (arXiv)

Finding new drugs is getting harder and harder. One of the hopes of drug discovery is to use machine learning models to predict molecular properties. That is why models for molecular property prediction are being developed and tested on benchmarks such as MoleculeNet. However, existing benchmarks are unrealistic and are too different from applying the models in practice. We have created a new practical Lo-Hi benchmark consisting of two tasks: Lead Optimization (Lo) and Hit Identification (Hi), corresponding to the real drug discovery process. For the Hi task, we designed a novel molecular splitting algorithm that solves the Balanced Vertex Minimum k-Cut problem. We tested state-of-the-art and classic ML models, revealing which works better under practical settings. We analyzed modern benchmarks and showed that they are unrealistic and overoptimistic. Review: https://openreview.net/forum?id=H2Yb28qGLV Lo-Hi benchmark: https://github.com/SteshinSS/lohineurips2023 Lo-Hi splitter library: https://github.com/SteshinSS/lohisplitter