# LeMat-GenBench: A Unified Evaluation Framework for Crystal Generative Models

arXiv: https://arxiv.org/abs/2512.04562 · HTML: https://mlsanity.com/paper/2512.04562/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 95 references that are in the map.
- First posted on arXiv 4 Dec 2025.

## Builds on (most cited first, 30 of 49)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation](https://mlsanity.com/paper/2106.04399/) (2021)
- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [GFlowNet Foundations](https://mlsanity.com/paper/2111.09266/) (2021)
- [Crystal Diffusion Variational Autoencoder for Periodic Material Generation](https://mlsanity.com/paper/2110.06197/) (2021)
- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Machine Learning Force Fields](https://mlsanity.com/paper/2010.07067/) (2020)
- [Fr'echet ChemNet Distance: A metric for generative models for molecules in drug discovery](https://mlsanity.com/paper/1803.09518/) (2018)
- [Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization](https://mlsanity.com/paper/2206.12411/) (2022)
- [Crystal Structure Prediction by Joint Equivariant Diffusion](https://mlsanity.com/paper/2309.04475/) (2023)
- [FlowMM: Generating Materials with Riemannian Flow Matching](https://mlsanity.com/paper/2406.04713/) (2024)
- [UMA: A Family of Universal Models for Atoms](https://mlsanity.com/paper/2506.23971/) (2025)
- [A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery](https://mlsanity.com/paper/2406.10833/) (2024)
- [Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations](https://mlsanity.com/paper/2210.07237/) (2022)
- [Fine-Tuned Language Models Generate Stable Inorganic Materials as Text](https://mlsanity.com/paper/2402.04379/) (2024)
- [All-atom Diffusion Transformers: Unified generative modelling of molecules and materials](https://mlsanity.com/paper/2503.03965/) (2025)
- [Space Group Constrained Crystal Generation](https://mlsanity.com/paper/2402.03992/) (2024)
- [A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems](https://mlsanity.com/paper/2312.07511/) (2023)
- [Generative Flow Networks as Entropy-Regularized RL](https://mlsanity.com/paper/2310.12934/) (2023)
- [Language models can generate molecules, materials, and protein binding sites directly in three dimensions as XYZ, CIF, and PDB files](https://mlsanity.com/paper/2305.05708/) (2023)
- [Scalable Diffusion for Materials Generation](https://mlsanity.com/paper/2311.09235/) (2023)

## Built on by (most cited first, 23 of 23)

- [Crystalite: A Lightweight Transformer for Efficient Crystal Modeling](https://mlsanity.com/paper/2604.02270/) (2026)
- [Open Materials Generation with Inference-Time Reinforcement Learning](https://mlsanity.com/paper/2602.00424/) (2026)
- [MADE: Benchmark Environments for Closed-Loop Materials Discovery](https://mlsanity.com/paper/2601.20996/) (2026)
- [Multimodal Crystal Flow: Any-to-Any Modality Generation for Unified Crystal Modeling](https://mlsanity.com/paper/2602.20210/) (2026)
- [Substitution-Based Analysis of Structural Novelty for Generative Models of Materials](https://mlsanity.com/paper/2606.23166/) (2026)
- [Transport Novelty Distance: A Distributional Metric for Evaluating Material Generative Models](https://mlsanity.com/paper/2512.09514/) (2025)
- [Flowing with Confidence](https://mlsanity.com/paper/2605.18472/) (2026)
- [MatMind: A Structure-Activity Knowledge-Driven Generative Foundation Model for Materials Science](https://mlsanity.com/paper/2606.07712/) (2026)
- [Surrogate-Gated Generation and Foundation-Model Embeddings for Bayesian Materials Design](https://mlsanity.com/paper/2606.28578/) (2026)
- [Chemical filters for ultra-high-throughput materials screening and generation](https://mlsanity.com/paper/2607.17910/) (2026)
- [OMatG-flash: An All-Atom Flow Map with Reinforce Adjoint Matching for Scalable Materials Discovery](https://mlsanity.com/paper/2609.26402/) (2026)
- [Inverse Design of Inorganic Compounds with Generative AI](https://mlsanity.com/paper/2604.11827/) (2026)
- [PRISMat: Policy-Driven, Permutation-Invariant Autoregressive Material Generation](https://mlsanity.com/paper/2605.16612/) (2026)
- [Atomistic Language Models Understand and Generate Materials](https://mlsanity.com/paper/2606.21395/) (2026)
- [Representations from Pretrained Machine-Learning Interatomic Potentials as Coarse Coordinates for Material Generation and Evaluation](https://mlsanity.com/paper/2607.28776/) (2026)
- [DynaCrys: Crystal Generation with Dynamic Space-Group Diffusion](https://mlsanity.com/paper/2608.07401/) (2026)
- [Autonomous discovery of new structure-plausibility laws for explainable and rapid crystal diagnosis and screening](https://mlsanity.com/paper/2609.01209/) (2026)
- [Deep Generative Crystal Structure Prediction: A Benchmark Study and a Controlled Test of Prototype Dependence](https://mlsanity.com/paper/2609.26502/) (2026)
- [Let CSP Be Your ANCHOR: Adaptive Crystal Search over Frozen Structure Priors](https://mlsanity.com/paper/2609.33407/) (2026)
- [GLASS: Global Latent Aggregation with Slot-based Set Decoding for Scalable All-Atom Crystal Generation](https://mlsanity.com/paper/2609.37158/) (2026)
- [Reinforcement Learning on the Discrete Composition Channel of a Crystal Generator: Validated Gains and Reward Hacking](https://mlsanity.com/paper/2610.03880/) (2026)
- [OxiGen: Oxidation-State-Aware Crystal Generation](https://mlsanity.com/paper/2610.08296/) (2026)
- [Zatom-2: Multitask Pretraining on Atomistic Data for Generative Modeling across Domains](https://mlsanity.com/paper/2610.11454/) (2026)

## Abstract (arXiv)

Generative machine learning (ML) models hold great promise for accelerating materials discovery through the inverse design of inorganic crystals, enabling an unprecedented exploration of chemical space. Yet, the lack of standardized evaluation frameworks makes it challenging to evaluate, compare, and further develop these ML models meaningfully. In this work, we introduce LeMat-GenBench, a unified benchmark for generative models of crystalline materials, supported by a set of evaluation metrics designed to better inform model development and downstream applications. We release both an open-source evaluation suite and a public leaderboard on Hugging Face, and benchmark 12 recent generative models. Results reveal that an increase in stability leads to a decrease in novelty and diversity on average, with no model excelling across all dimensions. Altogether, LeMat-GenBench establishes a reproducible and extensible foundation for fair model comparison and aims to guide the development of more reliable, discovery-oriented generative models for crystalline materials.