# Guiding Generative Models to Uncover Diverse and Novel Crystals via Reinforcement Learning

arXiv: https://arxiv.org/abs/2511.07158 · HTML: https://mlsanity.com/paper/2511.07158/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 63 references that are in the map.
- First posted on arXiv 10 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Crystal Diffusion Variational Autoencoder for Periodic Material Generation](https://mlsanity.com/paper/2110.06197/) (2021)
- [Crystal Structure Prediction by Joint Equivariant Diffusion](https://mlsanity.com/paper/2309.04475/) (2023)
- [FlowMM: Generating Materials with Riemannian Flow Matching](https://mlsanity.com/paper/2406.04713/) (2024)
- [Fine-Tuned Language Models Generate Stable Inorganic Materials as Text](https://mlsanity.com/paper/2402.04379/) (2024)
- [Large-scale Reinforcement Learning for Diffusion Models](https://mlsanity.com/paper/2401.12244/) (2024)
- [All-atom Diffusion Transformers: Unified generative modelling of molecules and materials](https://mlsanity.com/paper/2503.03965/) (2025)
- [Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2504.12501/) (2025)
- [Large-scale machine-learning-assisted exploration of the whole materials space](https://mlsanity.com/paper/2210.00579/) (2022)
- [Reinforcement Fine-Tuning for Materials Design](https://mlsanity.com/paper/2504.02367/) (2025)
- [CrystalGym: A New Benchmark for Materials Discovery Using Reinforcement Learning](https://mlsanity.com/paper/2509.23156/) (2025)

## Built on by (most cited first, 21 of 21)

- [Open Materials Generation with Inference-Time Reinforcement Learning](https://mlsanity.com/paper/2602.00424/) (2026)
- [MADE: Benchmark Environments for Closed-Loop Materials Discovery](https://mlsanity.com/paper/2601.20996/) (2026)
- [Control Variate Score Matching for Diffusion Models](https://mlsanity.com/paper/2512.20003/) (2025)
- [Zatom-1: Towards a Multimodal Foundation Model for 3D Molecules and Materials](https://mlsanity.com/paper/2602.22251/) (2026)
- [Generative Models for Crystalline Materials](https://mlsanity.com/paper/2511.22652/) (2025)
- [Crys-JEPA: Accelerating Crystal Discovery via Embedding Screening and Generative Refinement](https://mlsanity.com/paper/2605.14759/) (2026)
- [Substitution-Based Analysis of Structural Novelty for Generative Models of Materials](https://mlsanity.com/paper/2606.23166/) (2026)
- [Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputs](https://mlsanity.com/paper/2605.10385/) (2026)
- [MatMind: A Structure-Activity Knowledge-Driven Generative Foundation Model for Materials Science](https://mlsanity.com/paper/2606.07712/) (2026)
- [Autoregressive latent diffusion for 3D molecule generation](https://mlsanity.com/paper/2607.09277/) (2026)
- [Chemical filters for ultra-high-throughput materials screening and generation](https://mlsanity.com/paper/2607.17910/) (2026)
- [CrystalGRPO: Target-Aligned and Coverage-Preserving Reinforcement Learning for Flow-Based Crystal Structure Prediction](https://mlsanity.com/paper/2608.06582/) (2026)
- [Symmetry- and Property-Aware Crystal Generation with Reinforcement Learning for Inverse Materials Design](https://mlsanity.com/paper/2609.13468/) (2026)
- [OMatG-flash: An All-Atom Flow Map with Reinforce Adjoint Matching for Scalable Materials Discovery](https://mlsanity.com/paper/2609.26402/) (2026)
- [Representations from Pretrained Machine-Learning Interatomic Potentials as Coarse Coordinates for Material Generation and Evaluation](https://mlsanity.com/paper/2607.28776/) (2026)
- [On the Diversity of Analogy Making in Large Language Models](https://mlsanity.com/paper/2608.03233/) (2026)
- [Let CSP Be Your ANCHOR: Adaptive Crystal Search over Frozen Structure Priors](https://mlsanity.com/paper/2609.33407/) (2026)
- [GLASS: Global Latent Aggregation with Slot-based Set Decoding for Scalable All-Atom Crystal Generation](https://mlsanity.com/paper/2609.37158/) (2026)
- [Riemannian Flow Models with Reinforcement Learning for Molecular Crystal Structure Prediction](https://mlsanity.com/paper/2609.39773/) (2026)
- [Reinforcement Learning on the Discrete Composition Channel of a Crystal Generator: Validated Gains and Reward Hacking](https://mlsanity.com/paper/2610.03880/) (2026)
- [DiffGA: Steering Frozen Diffusion Models for Materials Design via Evolutionary Search](https://mlsanity.com/paper/2610.14999/) (2026)

## Abstract (arXiv)

Discovering functional crystalline materials entails navigating an immense combinatorial design space. While recent advances in generative artificial intelligence have enabled the sampling of chemically plausible compositions and structures, a fundamental challenge remains: the objective misalignment between likelihood-based sampling in generative modelling and targeted focus on underexplored regions where novel compounds reside. Here, we introduce a reinforcement learning framework that guides latent denoising diffusion models toward diverse and novel, yet thermodynamically viable crystalline compounds. Our approach integrates group relative policy optimisation with verifiable, multi-objective rewards that jointly balance creativity, stability, and diversity. Beyond de novo generation, we demonstrate enhanced property-guided design that preserves chemical validity, while targeting desired functional properties. This approach establishes a modular foundation for controllable AI-driven inverse design that addresses the novelty-validity trade-off across scientific discovery applications of generative models.