# Materials Discovery with Extreme Properties via Reinforcement Learning-Guided Combinatorial Chemistry

arXiv: https://arxiv.org/abs/2303.11833 · HTML: https://mlsanity.com/paper/2303.11833/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 7 references that are in the map.
- First posted on arXiv 21 Mar 2023.

## Built on by (most cited first, 5 of 5)

- [PriM: Principle-Inspired Material Discovery through Multi-Agent Collaboration](https://mlsanity.com/paper/2504.08810/) (2025)
- [AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Research](https://mlsanity.com/paper/2505.12039/) (2025)
- [Position: RL Researchers Need to Distinguish Between Solving Simulators and Using Simulators as a Proxy](https://mlsanity.com/paper/2606.28433/) (2026)
- [Machine-Learning-Assisted Photonic Device Development: A Multiscale Approach from Theory to Characterization](https://mlsanity.com/paper/2506.20056/) (2025)
- [Towards a Generalizable AI for Materials Discovery: Validation through Immersion Coolant Screening](https://mlsanity.com/paper/2510.23371/) (2025)

## Abstract (arXiv)

The goal of most materials discovery is to discover materials that are superior to those currently known. Fundamentally, this is close to extrapolation, which is a weak point for most machine learning models that learn the probability distribution of data. Herein, we develop reinforcement learning-guided combinatorial chemistry, which is a rule-based molecular designer driven by trained policy for selecting subsequent molecular fragments to get a target molecule. Since our model has the potential to generate all possible molecular structures that can be obtained from combinations of molecular fragments, unknown molecules with superior properties can be discovered. We theoretically and empirically demonstrate that our model is more suitable for discovering better compounds than probability distribution-learning models. In an experiment aimed at discovering molecules that hit seven extreme target properties, our model discovered 1,315 of all target-hitting molecules and 7,629 of five target-hitting molecules out of 100,000 trials, whereas the probability distribution-learning models failed. Moreover, it has been confirmed that every molecule generated under the binding rules of molecular fragments is 100% chemically valid. To illustrate the performance in actual problems, we also demonstrate that our models work well on two practical applications: discovering protein docking molecules and HIV inhibitors.