mlsanityData to 14 Oct 2026

All that structure matches does not glitter

arXiv 2509.12178 ↗First posted 15 Sep 2025cs.LG cond-mat.mtrl-scicited by 12

Not on the map. Why?

Abstract (arXiv)

Generative models for materials, especially inorganic crystals, hold potential to transform the theoretical prediction of novel compounds and structures. Advancement in this field depends on robust benchmarks and minimal, information-rich datasets that enable meaningful model evaluation. This paper critically examines common datasets and reported metrics for a crystal structure prediction taskunicodex2014generating the most likely structures given the chemical composition of a material. We focus on three key issues: First, materials datasets should contain unique crystal structures; for example, we show that the widely-utilized carbon-24 dataset only contains ≈40% unique structures. Second, materials datasets should not be split randomly if polymorphs of many different compositions are numerous, which we find to be the case for the perov-5 and MP-20 datasets. Third, benchmarks can mislead if used uncritically, e.g., reporting a match rate metric without considering the structural variety exhibited by identical building blocks. To address these oft-overlooked issues, we introduce several fixes. We provide revised versions of the carbon-24 dataset: one with duplicates removed, one deduplicated and split by number of atoms N, one with enantiomorphs, and two containing only identical structures but with different unit cells. We also propose new splits for datasets with polymorphs, ensuring that polymorphs are grouped within each split subset, setting a more sensible standard for benchmarking model performance. Finally, we present METRe and cRMSE, new model evaluation metrics that can correct existing issues with the match rate metric.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 28 of its 57 references that are in the map. First posted on arXiv 15 Sep 2025.

Builds on

28 of 57 references.

References · 28

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2022Flow Matching for Generative Modelingcited by 5,702
  4. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 939
  5. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 708
  6. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 474
  7. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  8. 2021Crystal Diffusion Variational Autoencoder for Periodic Material Generationcited by 270
  9. 2023Flow Matching on General Geometriescited by 265
  10. 2021How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Modelscited by 196
  11. 2023Crystal Structure Prediction by Joint Equivariant Diffusioncited by 148
  12. 2024FlowMM: Generating Materials with Riemannian Flow Matchingcited by 109
16 more
  1. 2023Stochastic interpolants with data-dependent couplingscited by 100
  2. 2018An empirical study on evaluation metrics of generative adversarial networkscited by 81
  3. 2025All-atom Diffusion Transformers: Unified generative modelling of molecules and materialscited by 75
  4. 2024Space Group Constrained Crystal Generationcited by 73
  5. 2024FlowLLM: Flow Matching for Material Generation with Large Language Models as Base Distributionscited by 53
  6. 2025Open Materials Generation with Stochastic Interpolantscited by 28
  7. 2025A Periodic Bayesian Flow for Material Generationcited by 16
  8. 2025Periodic Materials Generation using Text-Guided Joint Diffusion Modelcited by 15
  9. 2022Large-scale machine-learning-assisted exploration of the whole materials spacecited by 13
  10. 2023Equivariant Message Passing Neural Network for Crystal Material Discoverycited by 10
  11. 2025Establishing baselines for generative discovery of inorganic crystalscited by 9
  12. 2025Kinetic Langevin Diffusion for Crystalline Materials Generationcited by 9
  13. 2024Generative Inverse Design of Crystal Structures via Diffusion Models with Transformerscited by 7
  14. 2025CrystalGRW: Generative Modeling of Crystal Structures with Targeted Properties via Geodesic Random Walkscited by 5
  15. 2025Transformer-Enhanced Variational Autoencoder for Crystal Structure Predictioncited by 2
  16. 2025ContinuouSP: Generative Model for Crystal Structure Prediction with Invariance and Continuitycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 11.

12 papers.

  1. 2026Open Materials Generation with Inference-Time Reinforcement Learningcited by 8
  2. 2026Multimodal Crystal Flow: Any-to-Any Modality Generation for Unified Crystal Modelingcited by 7
  3. 2025Generative Models for Crystalline Materialscited by 4
  4. 2026MEIDNet: Multimodal generative AI framework for inverse materials designcited by 1
  5. 2026XRDiff: Crystal Structure Prediction from Powder X-Ray Diffraction Data Using Diffusion Modelscited by 1
  6. 2026Discovering Crystal Structure Prediction Algorithms with an AI Co-Scientistcited by 1
  7. 2026CrystalGRPO: Target-Aligned and Coverage-Preserving Reinforcement Learning for Flow-Based Crystal Structure Predictioncited by 1
  8. 2026OMatG-flash: An All-Atom Flow Map with Reinforce Adjoint Matching for Scalable Materials Discoverycited by 1
  9. 2026Atomistic Language Models Understand and Generate Materialscited by 0
  10. 2026uFlowCSP: Crystal Structure Prediction using Mean flow generative modelscited by 0
  11. 2026Riemannian Flow Models with Reinforcement Learning for Molecular Crystal Structure Predictioncited by 0
  12. 2026Reinforcement Learning on the Discrete Composition Channel of a Crystal Generator: Validated Gains and Reward Hackingcited by 0