# Discovery of 2D materials using Transformer Network based Generative Design

arXiv: https://arxiv.org/abs/2301.05824 · HTML: https://mlsanity.com/paper/2301.05824/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 57 references that are in the map.
- First posted on arXiv 14 Jan 2023.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Crystal Diffusion Variational Autoencoder for Periodic Material Generation](https://mlsanity.com/paper/2110.06197/) (2021)
- [Blank Language Models](https://mlsanity.com/paper/2002.03079/) (2020)
- [Data-driven discovery of novel 2D materials by deep generative models](https://mlsanity.com/paper/2206.12159/) (2022)
- [Crystal structure prediction with machine learning-based element substitution](https://mlsanity.com/paper/2201.11188/) (2022)
- [Crystal Transformer: Self-learning neural language model for Generative and Tinkering Design of Materials](https://mlsanity.com/paper/2204.11953/) (2022)
- [Machine-learning accelerated identification of exfoliable two-dimensional materials](https://mlsanity.com/paper/2207.12118/) (2022)

## Built on by (most cited first, 5 of 5)

- [TopoMAS: Large Language Model Driven Topological Materials Multiagent System](https://mlsanity.com/paper/2507.04053/) (2025)
- [Artificial Intelligence in Materials Science and Engineering: Current Landscape, Key Challenges, and Future Trajectorie](https://mlsanity.com/paper/2601.12554/) (2026)
- [Discovery of 2D Materials via Symmetry-Constrained Diffusion Model](https://mlsanity.com/paper/2412.18414/) (2024)
- [SLayerGen: a Crystal Generative Model for all Space and Layer Groups](https://mlsanity.com/paper/2605.08262/) (2026)
- [Enhancing Large Language Models with Domain-Specific Knowledge: The Case in Topological Materials](https://mlsanity.com/paper/2409.13732/) (2024)

## Abstract (arXiv)

Two-dimensional (2D) materials have wide applications in superconductors, quantum, and topological materials. However, their rational design is not well established, and currently less than 6,000 experimentally synthesized 2D materials have been reported. Recently, deep learning, data-mining, and density functional theory (DFT)-based high-throughput calculations are widely performed to discover potential new materials for diverse applications. Here we propose a generative material design pipeline, namely material transformer generator(MTG), for large-scale discovery of hypothetical 2D materials. We train two 2D materials composition generators using self-learning neural language models based on Transformers with and without transfer learning. The models are then used to generate a large number of candidate 2D compositions, which are fed to known 2D materials templates for crystal structure prediction. Next, we performed DFT computations to study their thermodynamic stability based on energy-above-hull and formation energy. We report four new DFT-verified stable 2D materials with zero e-above-hull energies, including NiCl4, IrSBr, CuBr3, and CoBrCl. Our work thus demonstrates the potential of our MTG generative materials design pipeline in the discovery of novel 2D materials and other functional materials.