# Space Group Informed Transformer for Crystalline Materials Generation

arXiv: https://arxiv.org/abs/2403.15734 · HTML: https://mlsanity.com/paper/2403.15734/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 137 references that are in the map.
- First posted on arXiv 23 Mar 2024.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Crystal Diffusion Variational Autoencoder for Periodic Material Generation](https://mlsanity.com/paper/2110.06197/) (2021)
- [Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules](https://mlsanity.com/paper/1906.00957/) (2019)
- [COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics](https://mlsanity.com/paper/2202.11705/) (2022)
- [Crystal Structure Prediction by Joint Equivariant Diffusion](https://mlsanity.com/paper/2309.04475/) (2023)
- [MatterGen: a generative model for inorganic materials design](https://mlsanity.com/paper/2312.03687/) (2023)
- [CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling](https://mlsanity.com/paper/1811.10996/) (2018)
- [Inverse design of 3d molecular structures with conditional generative neural networks](https://mlsanity.com/paper/2109.04824/) (2021)
- [Fine-Tuned Language Models Generate Stable Inorganic Materials as Text](https://mlsanity.com/paper/2402.04379/) (2024)
- [Periodic Graph Transformers for Crystal Material Property Prediction](https://mlsanity.com/paper/2209.11807/) (2022)
- [Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs](https://mlsanity.com/paper/2306.03081/) (2023)
- [Targeted free energy estimation via learned mappings](https://mlsanity.com/paper/2002.04913/) (2020)
- [Language models can generate molecules, materials, and protein binding sites directly in three dimensions as XYZ, CIF, and PDB files](https://mlsanity.com/paper/2305.05708/) (2023)
- [An invertible crystallographic representation for general inverse design of inorganic crystals with targeted properties](https://mlsanity.com/paper/2005.07609/) (2020)
- [Normalizing flows for atomic solids](https://mlsanity.com/paper/2111.08696/) (2021)
- [Crystal-GFN: sampling crystals with desirable properties and constraints](https://mlsanity.com/paper/2310.04925/) (2023)
- [LLM-Prop: Predicting Physical And Electronic Properties Of Crystalline Solids From Their Text Descriptions](https://mlsanity.com/paper/2310.14029/) (2023)
- [Rigid Body Flows for Sampling Molecular Crystal Structures](https://mlsanity.com/paper/2301.11355/) (2023)
- [Towards Predicting Equilibrium Distributions for Molecular Systems with Deep Learning](https://mlsanity.com/paper/2306.05445/) (2023)
- [Towards Symmetry-Aware Generation of Periodic Materials](https://mlsanity.com/paper/2307.02707/) (2023)
- [Large-scale machine-learning-assisted exploration of the whole materials space](https://mlsanity.com/paper/2210.00579/) (2022)
- [Generating equilibrium molecules with deep neural networks](https://mlsanity.com/paper/1810.11347/) (2018)
- [End-to-End Crystal Structure Prediction from Powder X-Ray Diffraction](https://mlsanity.com/paper/2401.03862/) (2024)

## Built on by (most cited first, 30 of 41)

- [SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models](https://mlsanity.com/paper/2502.03638/) (2025)
- [Open Materials Generation with Stochastic Interpolants](https://mlsanity.com/paper/2502.02582/) (2025)
- [Wyckoff Transformer: Generation of Symmetric Crystals](https://mlsanity.com/paper/2503.02407/) (2025)
- [LeMat-GenBench: A Unified Evaluation Framework for Crystal Generative Models](https://mlsanity.com/paper/2512.04562/) (2025)
- [A Periodic Bayesian Flow for Material Generation](https://mlsanity.com/paper/2502.02016/) (2025)
- [PLaID++: A Preference Aligned Language Model for Targeted Inorganic Materials Design](https://mlsanity.com/paper/2509.07150/) (2025)
- [Reinforcement Fine-Tuning for Materials Design](https://mlsanity.com/paper/2504.02367/) (2025)
- [AI-driven inverse design of materials: Past, present and future](https://mlsanity.com/paper/2411.09429/) (2024)
- [Crystalite: A Lightweight Transformer for Efficient Crystal Modeling](https://mlsanity.com/paper/2604.02270/) (2026)
- [Symmetry-Aware Bayesian Flow Networks for Crystal Generation](https://mlsanity.com/paper/2502.03146/) (2025)
- [Artificial Intelligence and Generative Models for Materials Discovery -- A Review](https://mlsanity.com/paper/2508.03278/) (2025)
- [A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Tools](https://mlsanity.com/paper/2506.20743/) (2025)
- [Space Group Conditional Flow Matching](https://mlsanity.com/paper/2509.23822/) (2025)
- [Multimodal Crystal Flow: Any-to-Any Modality Generation for Unified Crystal Modeling](https://mlsanity.com/paper/2602.20210/) (2026)
- [TopoMAS: Large Language Model Driven Topological Materials Multiagent System](https://mlsanity.com/paper/2507.04053/) (2025)
- [Zatom-1: Towards a Multimodal Foundation Model for 3D Molecules and Materials](https://mlsanity.com/paper/2602.22251/) (2026)
- [InvDesFlow-AL: active learning-based workflow for inverse design of functional materials](https://mlsanity.com/paper/2505.09203/) (2025)
- [MiAD: Mirage Atom Diffusion for De Novo Crystal Generation](https://mlsanity.com/paper/2511.14426/) (2025)
- [Generative Models for Crystalline Materials](https://mlsanity.com/paper/2511.22652/) (2025)
- [PhononBench:A Large-Scale Phonon-Based Benchmark for Dynamical Stability in Crystal Generation](https://mlsanity.com/paper/2512.21227/) (2025)
- [AI-Assisted Rapid Crystal Structure Generation Towards a Target Local Environment](https://mlsanity.com/paper/2506.08224/) (2025)
- [CrystalICL: Enabling In-Context Learning for Crystal Generation](https://mlsanity.com/paper/2508.20143/) (2025)
- [A Generation Framework with Strict Constraints for Crystal Materials Design](https://mlsanity.com/paper/2411.08464/) (2024)
- [CrystalFormer-CSP: Thinking Fast and Slow for Crystal Structure Prediction](https://mlsanity.com/paper/2512.18251/) (2025)
- [Fourier Transformers for Latent Crystallographic Diffusion and Generative Modeling](https://mlsanity.com/paper/2602.12045/) (2026)
- [Composable Crystals: Controllable Materials Discovery via Concept Learning](https://mlsanity.com/paper/2605.14769/) (2026)
- [MatMind: A Structure-Activity Knowledge-Driven Generative Foundation Model for Materials Science](https://mlsanity.com/paper/2606.07712/) (2026)
- [Discovering Crystal Structure Prediction Algorithms with an AI Co-Scientist](https://mlsanity.com/paper/2606.22866/) (2026)
- [Structural Constraint Integration in Generative Model for Discovery of Quantum Material Candidates](https://mlsanity.com/paper/2407.04557/) (2024)
- [Design Topological Materials by Reinforcement Fine-Tuned Generative Model](https://mlsanity.com/paper/2504.13048/) (2025)

## Abstract (arXiv)

We introduce CrystalFormer, a transformer-based autoregressive model specifically designed for space group-controlled generation of crystalline materials. By explicitly incorporating space group symmetry, CrystalFormer greatly reduces the effective complexity of crystal space, which is essential for data-and compute-efficient generative modeling of crystalline materials. Leveraging the prominent discrete and sequential nature of the Wyckoff positions, CrystalFormer learns to generate crystals by directly predicting the species and coordinates of symmetry-inequivalent atoms in the unit cell. We demonstrate the advantages of CrystalFormer in standard tasks such as symmetric structure initialization and element substitution over widely used conventional approaches. Furthermore, we showcase its plug-and-play application to property-guided materials design, highlighting its flexibility. Our analysis reveals that CrystalFormer ingests sensible solid-state chemistry knowledge and heuristics by compressing the material dataset, thus enabling systematic exploration of crystalline materials space. The simplicity, generality, and adaptability of CrystalFormer position it as a promising architecture to be the foundational model of the entire crystalline materials space, heralding a new era in materials discovery and design.