# Deep Reinforcement Learning for Inverse Inorganic Materials Design

arXiv: https://arxiv.org/abs/2210.11931 · HTML: https://mlsanity.com/paper/2210.11931/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 119 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Deep Reinforcement Learning for De-Novo Drug Design](https://mlsanity.com/paper/1711.10907/) (2017)
- [Continuous Deep Q-Learning with Model-based Acceleration](https://mlsanity.com/paper/1603.00748/) (2016)
- [Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets](https://mlsanity.com/paper/1503.01007/) (2015)
- [Molecular generative model based on conditional variational autoencoder for de novo molecular design](https://mlsanity.com/paper/1806.05805/) (2018)
- [Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines](https://mlsanity.com/paper/1803.07246/) (2018)
- [Predicting materials properties without crystal structure: Deep representation learning from stoichiometry](https://mlsanity.com/paper/1910.00617/) (2019)
- [Generative adversarial networks (GAN) based efficient sampling of chemical space for inverse design of inorganic materials](https://mlsanity.com/paper/1911.05020/) (2019)
- [Crystal structure prediction with machine learning-based element substitution](https://mlsanity.com/paper/2201.11188/) (2022)
- [Fast Bayesian Optimization of Needle-in-a-Haystack Problems using Zooming Memory-Based Initialization (ZoMBI)](https://mlsanity.com/paper/2208.13771/) (2022)
- [Computational discovery of new 2D materials using deep learning generative models](https://mlsanity.com/paper/2012.09314/) (2020)
- [Constrained Model-Free Reinforcement Learning for Process Optimization](https://mlsanity.com/paper/2011.07925/) (2020)
- [CRYSPNet: Crystal Structure Predictions via Neural Network](https://mlsanity.com/paper/2003.14328/) (2020)
- [Deep Reinforcement Learning Optimizes Graphene Nanopores for Efficient Desalination](https://mlsanity.com/paper/2101.07399/) (2021)
- [Learning Operations on a Stack with Neural Turing Machines](https://mlsanity.com/paper/1612.00827/) (2016)

## Built on by (most cited first, 19 of 19)

- [Artificial Intelligence and Generative Models for Materials Discovery -- A Review](https://mlsanity.com/paper/2508.03278/) (2025)
- [Cross-Modal Learning for Chemistry Property Prediction: Large Language Models Meet Graph Machine Learning](https://mlsanity.com/paper/2408.14964/) (2024)
- [Generative Models for Crystalline Materials](https://mlsanity.com/paper/2511.22652/) (2025)
- [Towards Agentic Intelligence for Materials Science](https://mlsanity.com/paper/2602.00169/) (2026)
- [VAE for Modified 1-Hot Generative Materials Modeling, A Step Towards Inverse Material Design](https://mlsanity.com/paper/2401.06779/) (2023)
- [DiffSyn: A Generative Diffusion Approach to Materials Synthesis Planning](https://mlsanity.com/paper/2509.17094/) (2025)
- [MatMind: A Structure-Activity Knowledge-Driven Generative Foundation Model for Materials Science](https://mlsanity.com/paper/2606.07712/) (2026)
- [Adaptable Method for Crystal Design across Diverse Constraints and Objectives with Pretrained Property Predictors](https://mlsanity.com/paper/2410.08562/) (2024)
- [MACS: Multi-Agent Reinforcement Learning for Optimization of Crystal Structures](https://mlsanity.com/paper/2506.04195/) (2025)
- [AIMatDesign: Knowledge-Augmented Reinforcement Learning for Inverse Materials Design under Data Scarcity](https://mlsanity.com/paper/2507.00024/) (2025)
- [Unified machine-learning framework for property prediction and time-evolution simulation of strained alloy microstructure](https://mlsanity.com/paper/2507.21760/) (2025)
- [Extended Factorization Machine Annealing for Rapid Discovery of Transparent Conducting Materials](https://mlsanity.com/paper/2507.23160/) (2025)
- [Philosophy-informed Machine Learning](https://mlsanity.com/paper/2509.20370/) (2025)
- [Neural surrogates for crystal growth dynamics with variable supersaturation: explicit vs. implicit conditioning](https://mlsanity.com/paper/2604.21753/) (2026)
- [A Padding Method for Enhanced Encoding of Inorganic Structures with Varying Chemical Compositions](https://mlsanity.com/paper/2605.30743/) (2026)
- [Towards Automated Discovery: A Review of Generative Models, Multimodal Learning and Closed-Loop Workflows in Inverse Materials Design](https://mlsanity.com/paper/2606.02507/) (2026)
- [Let CSP Be Your ANCHOR: Adaptive Crystal Search over Frozen Structure Priors](https://mlsanity.com/paper/2609.33407/) (2026)
- [Riemannian Flow Models with Reinforcement Learning for Molecular Crystal Structure Prediction](https://mlsanity.com/paper/2609.39773/) (2026)
- [Reinforcement Learning on the Discrete Composition Channel of a Crystal Generator: Validated Gains and Reward Hacking](https://mlsanity.com/paper/2610.03880/) (2026)

## Abstract (arXiv)

A major obstacle to the realization of novel inorganic materials with desirable properties is the inability to perform efficient optimization across both materials properties and synthesis of those materials. In this work, we propose a reinforcement learning (RL) approach to inverse inorganic materials design, which can identify promising compounds with specified properties and synthesizability constraints. Our model learns chemical guidelines such as charge and electronegativity neutrality while maintaining chemical diversity and uniqueness. We demonstrate a multi-objective RL approach, which can generate novel compounds with targeted materials properties including formation energy and bulk/shear modulus alongside a lower sintering temperature synthesis objectives. Using this approach, the model can predict promising compounds of interest, while suggesting an optimized chemical design space for inorganic materials discovery.