# Generative Design of inorganic compounds using deep diffusion language models

arXiv: https://arxiv.org/abs/2310.00475 · HTML: https://mlsanity.com/paper/2310.00475/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 30 Sep 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](https://mlsanity.com/paper/2203.02923/) (2022)
- [Crystal Diffusion Variational Autoencoder for Periodic Material Generation](https://mlsanity.com/paper/2110.06197/) (2021)
- [DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models](https://mlsanity.com/paper/2211.15029/) (2022)
- [Learning Gradient Fields for Molecular Conformation Generation](https://mlsanity.com/paper/2105.03902/) (2021)
- [Generative adversarial networks (GAN) based efficient sampling of chemical space for inverse design of inorganic materials](https://mlsanity.com/paper/1911.05020/) (2019)
- [Data-driven discovery of novel 2D materials by deep generative models](https://mlsanity.com/paper/2206.12159/) (2022)
- [Crystal structure prediction with machine learning-based element substitution](https://mlsanity.com/paper/2201.11188/) (2022)
- [Crystal Transformer: Self-learning neural language model for Generative and Tinkering Design of Materials](https://mlsanity.com/paper/2204.11953/) (2022)
- [Crystal structure prediction using neural network potential and age-fitness Pareto genetic algorithm](https://mlsanity.com/paper/2309.06710/) (2023)
- [Composition based oxidation state prediction of materials using deep learning](https://mlsanity.com/paper/2211.15895/) (2022)

## Built on by (most cited first, 2 of 2)

- [Artificial Intelligence and Generative Models for Materials Discovery -- A Review](https://mlsanity.com/paper/2508.03278/) (2025)
- [Constrained Diffusion for Accelerated Structure Relaxation of Inorganic Solids with Point Defects](https://mlsanity.com/paper/2602.19153/) (2026)

## Abstract (arXiv)

Due to the vast chemical space, discovering materials with a specific function is challenging. Chemical formulas are obligated to conform to a set of exacting criteria such as charge neutrality, balanced electronegativity, synthesizability, and mechanical stability. In response to this formidable task, we introduce a deep learning-based generative model for material composition and structure design by learning and exploiting explicit and implicit chemical knowledge. Our pipeline first uses deep diffusion language models as the generator of compositions and then applies a template-based crystal structure prediction algorithm to predict their corresponding structures, which is then followed by structure relaxation using a universal graph neural network-based potential. The density functional theory (DFT) calculations of the formation energies and energy-above-the-hull analysis are used to validate new structures generated through our pipeline. Based on the DFT calculation results, six new materials, including Ti2HfO5, TaNbP, YMoN2, TaReO4, HfTiO2, and HfMnO2, with formation energy less than zero have been found. Remarkably, among these, four materials, namely Ti2HfO5, TaNbP, YMoN2, and TaReO4, exhibit an e-above-hull energy of less than 0.3 eV. These findings have proved the effectiveness of our approach.