Crys-JEPA: Accelerating Crystal Discovery via Embedding Screening and Generative Refinement
Not on the map. Why?
Abstract (arXiv)
De novo crystal generation seeks to discover materials that are not merely realistic, but also stable and novel. However, most existing generative models are trained to maximize the likelihood of observed crystals, which encourages samples to stay close to known materials yet not necessarily align with the criteria that matter in discovery. Our empirical analysis shows that current crystal generative models exhibit a clear conflict between stability and novelty: samples near the observed distribution tend to retain stability but offer limited novelty, whereas samples farther from it often lose stability rapidly. This suggests that the useful region for discovering crystals that are both stable and novel is extremely narrow. To move beyond this limitation, we introduce Crys-JEPA, a joint embedding predictive architecture for crystals that learns an energy-aware latent space preserving formation-energy differences. In this space, stability assessment can be reformulated as an embedding-based comparison against accessible training crystals, reducing the reliance on expensive energy evaluation and task-specific external references. Building on Crys-JEPA, we further develop a screening-and-refinement pipeline that identifies promising generated crystals and reintroduces them to refine the generative model. On MP-20 and Alex-MP-20 datasets, we achieve improvements over baselines up to 53.8% and 72.7% on V.S.U.N. metric, respectively.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 40 of its 61 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
40 of 61 references.
References · 40
- 2017Attention Is All You Needcited by 59,932
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
- 2016Gaussian Error Linear Units (GELUs)cited by 3,431
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
28 more
- 2020On Layer Normalization in the Transformer Architecturecited by 906
- 2020A Generalization of Transformer Networks to Graphscited by 653
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
- 2021Crystal Diffusion Variational Autoencoder for Periodic Material Generationcited by 268
- 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
- 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 227
- 2023Geometric Latent Diffusion Models for 3D Molecule Generationcited by 185
- 2023Crystal Structure Prediction by Joint Equivariant Diffusioncited by 142
- 2024FlowMM: Generating Materials with Riemannian Flow Matchingcited by 105
- 2023Specformer: Spectral Graph Neural Networks Meet Transformerscited by 102
- 2023MatterGen: a generative model for inorganic materials designcited by 98
- 2024Fine-Tuned Language Models Generate Stable Inorganic Materials as Textcited by 92
- 2025Learning Smooth and Expressive Interatomic Potentials for Physical Property Predictioncited by 79
- 2025All-atom Diffusion Transformers: Unified generative modelling of molecules and materialscited by 75
- 2024Space Group Constrained Crystal Generationcited by 71
- 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 61
- 2024FlowLLM: Flow Matching for Material Generation with Large Language Models as Base Distributionscited by 52
- 2025SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Modelscited by 48
- 2025LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architecturescited by 35
- 2025Guiding Generative Models to Uncover Diverse and Novel Crystals via Reinforcement Learningcited by 19
- 2025Space Group Equivariant Crystal Diffusioncited by 15
- 2025LLM Meets Diffusion: A Hybrid Framework for Crystal Material Generationcited by 12
- 2025Establishing baselines for generative discovery of inorganic crystalscited by 8
- 2026Toward Graph-Tokenizing Large Language Models with Reconstructive Graph Instruction Tuningcited by 7
- 2025CrystalDiT: A Diffusion Transformer for Crystal Generationcited by 6
- 2026Conversation for Non-verifiable Learning: Self-Evolving LLMs through Meta-Evaluationcited by 5
- 2025Introducing physics-informed generative models for targeting structural novelty in the exploration of chemical spacecited by 2
- 2026Graph-GRPO: Training Graph Flow Models with Reinforcement Learningcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.