# All that structure matches does not glitter

arXiv: https://arxiv.org/abs/2509.12178 · HTML: https://mlsanity.com/paper/2509.12178/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 57 references that are in the map.
- First posted on arXiv 15 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [Crystal Diffusion Variational Autoencoder for Periodic Material Generation](https://mlsanity.com/paper/2110.06197/) (2021)
- [Flow Matching on General Geometries](https://mlsanity.com/paper/2302.03660/) (2023)
- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [Crystal Structure Prediction by Joint Equivariant Diffusion](https://mlsanity.com/paper/2309.04475/) (2023)
- [FlowMM: Generating Materials with Riemannian Flow Matching](https://mlsanity.com/paper/2406.04713/) (2024)
- [Stochastic interpolants with data-dependent couplings](https://mlsanity.com/paper/2310.03725/) (2023)
- [An empirical study on evaluation metrics of generative adversarial networks](https://mlsanity.com/paper/1806.07755/) (2018)
- [All-atom Diffusion Transformers: Unified generative modelling of molecules and materials](https://mlsanity.com/paper/2503.03965/) (2025)
- [Space Group Constrained Crystal Generation](https://mlsanity.com/paper/2402.03992/) (2024)
- [FlowLLM: Flow Matching for Material Generation with Large Language Models as Base Distributions](https://mlsanity.com/paper/2410.23405/) (2024)
- [Open Materials Generation with Stochastic Interpolants](https://mlsanity.com/paper/2502.02582/) (2025)
- [A Periodic Bayesian Flow for Material Generation](https://mlsanity.com/paper/2502.02016/) (2025)
- [Periodic Materials Generation using Text-Guided Joint Diffusion Model](https://mlsanity.com/paper/2503.00522/) (2025)
- [Large-scale machine-learning-assisted exploration of the whole materials space](https://mlsanity.com/paper/2210.00579/) (2022)
- [Equivariant Message Passing Neural Network for Crystal Material Discovery](https://mlsanity.com/paper/2302.00485/) (2023)
- [Establishing baselines for generative discovery of inorganic crystals](https://mlsanity.com/paper/2501.02144/) (2025)
- [Kinetic Langevin Diffusion for Crystalline Materials Generation](https://mlsanity.com/paper/2507.03602/) (2025)
- [Generative Inverse Design of Crystal Structures via Diffusion Models with Transformers](https://mlsanity.com/paper/2406.09263/) (2024)
- [CrystalGRW: Generative Modeling of Crystal Structures with Targeted Properties via Geodesic Random Walks](https://mlsanity.com/paper/2501.08998/) (2025)
- [Transformer-Enhanced Variational Autoencoder for Crystal Structure Prediction](https://mlsanity.com/paper/2502.09423/) (2025)
- [ContinuouSP: Generative Model for Crystal Structure Prediction with Invariance and Continuity](https://mlsanity.com/paper/2502.02026/) (2025)

## Built on by (most cited first, 12 of 12)

- [Open Materials Generation with Inference-Time Reinforcement Learning](https://mlsanity.com/paper/2602.00424/) (2026)
- [Multimodal Crystal Flow: Any-to-Any Modality Generation for Unified Crystal Modeling](https://mlsanity.com/paper/2602.20210/) (2026)
- [Generative Models for Crystalline Materials](https://mlsanity.com/paper/2511.22652/) (2025)
- [MEIDNet: Multimodal generative AI framework for inverse materials design](https://mlsanity.com/paper/2601.22009/) (2026)
- [XRDiff: Crystal Structure Prediction from Powder X-Ray Diffraction Data Using Diffusion Models](https://mlsanity.com/paper/2606.14003/) (2026)
- [Discovering Crystal Structure Prediction Algorithms with an AI Co-Scientist](https://mlsanity.com/paper/2606.22866/) (2026)
- [CrystalGRPO: Target-Aligned and Coverage-Preserving Reinforcement Learning for Flow-Based Crystal Structure Prediction](https://mlsanity.com/paper/2608.06582/) (2026)
- [OMatG-flash: An All-Atom Flow Map with Reinforce Adjoint Matching for Scalable Materials Discovery](https://mlsanity.com/paper/2609.26402/) (2026)
- [Atomistic Language Models Understand and Generate Materials](https://mlsanity.com/paper/2606.21395/) (2026)
- [uFlowCSP: Crystal Structure Prediction using Mean flow generative models](https://mlsanity.com/paper/2609.09799/) (2026)
- [Riemannian Flow Models with Reinforcement Learning for Molecular Crystal Structure Prediction](https://mlsanity.com/paper/2609.39773/) (2026)
- [Reinforcement Learning on the Discrete Composition Channel of a Crystal Generator: Validated Gains and Reward Hacking](https://mlsanity.com/paper/2610.03880/) (2026)

## Abstract (arXiv)

Generative models for materials, especially inorganic crystals, hold potential to transform the theoretical prediction of novel compounds and structures. Advancement in this field depends on robust benchmarks and minimal, information-rich datasets that enable meaningful model evaluation. This paper critically examines common datasets and reported metrics for a crystal structure prediction taskunicodex2014generating the most likely structures given the chemical composition of a material. We focus on three key issues: First, materials datasets should contain unique crystal structures; for example, we show that the widely-utilized carbon-24 dataset only contains ≈40% unique structures. Second, materials datasets should not be split randomly if polymorphs of many different compositions are numerous, which we find to be the case for the perov-5 and MP-20 datasets. Third, benchmarks can mislead if used uncritically, e.g., reporting a match rate metric without considering the structural variety exhibited by identical building blocks. To address these oft-overlooked issues, we introduce several fixes. We provide revised versions of the carbon-24 dataset: one with duplicates removed, one deduplicated and split by number of atoms N, one with enantiomorphs, and two containing only identical structures but with different unit cells. We also propose new splits for datasets with polymorphs, ensuring that polymorphs are grouped within each split subset, setting a more sensible standard for benchmarking model performance. Finally, we present METRe and cRMSE, new model evaluation metrics that can correct existing issues with the match rate metric.