mlsanityData to 14 Oct 2026

Reinforcement Fine-Tuning for Materials Design

arXiv 2504.02367 ↗First posted 3 Apr 2025cond-mat.mtrl-sci cs.LG physics.comp-phcited by 13

Not on the map. Why?

Abstract (arXiv)

Reinforcement fine-tuning played an instrumental role in enhancing the instruction-following and reasoning abilities of large language models. In this work, we employ reinforcement fine-tuning for materials design, in which discriminative machine learning models are used to provide rewards to the autoregressive transformer-based materials generative model CrystalFormer. By optimizing the reward signals-such as energy above the convex hull and material properties figures of merit-reinforcement fine-tuning infuses knowledge from discriminative models into generative models. The resulting model, CrystalFormer-RL, shows enhanced stability in generated crystals and successfully discovers crystals with desirable yet conflicting material properties, such as substantial dielectric constant and band gap simultaneously. Notably, we observe that reinforcement fine-tuning not only enables the property-guided material design but also unlocks property-based material retrieval behavior of pretrained generative model. The present framework opens an exciting gateway to the synergies of the machine learning ecosystem for materials design.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 35 of its 81 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

35 of 81 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  10. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  11. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  12. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
23 more
  1. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  2. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 708
  3. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  4. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 474
  5. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 283
  6. 2021Crystal Diffusion Variational Autoencoder for Periodic Material Generationcited by 270
  7. 2017Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanicscited by 214
  8. 2020Machine Learning Force Fieldscited by 207
  9. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 173
  10. 2017Deep Reinforcement Learning for De-Novo Drug Designcited by 161
  11. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 135
  12. 2016Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-controlcited by 101
  13. 2018Solving Statistical Mechanics Using Variational Autoregressive Networkscited by 100
  14. 2024Fine-Tuned Language Models Generate Stable Inorganic Materials as Textcited by 93
  15. 2025Learning Smooth and Expressive Interatomic Potentials for Physical Property Predictioncited by 84
  16. 2023Optimizing DDPM Sampling with Shortcut Fine-Tuningcited by 81
  17. 2024Orb: A Fast, Scalable Neural Network Potentialcited by 66
  18. 2023Language models can generate molecules, materials, and protein binding sites directly in three dimensions as XYZ, CIF, and PDB filescited by 59
  19. 2024Space Group Informed Transformer for Crystalline Materials Generationcited by 41
  20. 2023Matbench Discovery -- A framework to evaluate machine learning crystal stability predictionscited by 35
  21. 2023CHGNet: Pretrained universal neural network potential for charge-informed atomistic modelingcited by 13
  22. 2024InvDesFlow: An AI-driven materials inverse design workflow to explore possible high-temperature superconductorscited by 6
  23. 2025Active Learning for Conditional Inverse Design with Crystal Generation and Foundation Atomic Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 7.

13 papers.

Sort:

By area:

  1. 2025Guiding Generative Models to Uncover Diverse and Novel Crystals via Reinforcement Learningcited by 21
  2. 2025Artificial Intelligence and Generative Models for Materials Discovery -- A Reviewcited by 8
  3. 2025TopoMAS: Large Language Model Driven Topological Materials Multiagent Systemcited by 6
  4. 2026DMFlow: Disordered Materials Generation by Flow Matchingcited by 5
  5. 2025InvDesFlow-AL: active learning-based workflow for inverse design of functional materialscited by 4
  6. 2025PhononBench:A Large-Scale Phonon-Based Benchmark for Dynamical Stability in Crystal Generationcited by 3
  7. 2025CrystalFormer-CSP: Thinking Fast and Slow for Crystal Structure Predictioncited by 1
  8. 2026MatMind: A Structure-Activity Knowledge-Driven Generative Foundation Model for Materials Sciencecited by 1
  9. 2026Surrogate-Gated Generation and Foundation-Model Embeddings for Bayesian Materials Designcited by 1
  10. 2026Symmetry- and Property-Aware Crystal Generation with Reinforcement Learning for Inverse Materials Designcited by 1
  11. 2026Riemannian Flow Models with Reinforcement Learning for Molecular Crystal Structure Predictioncited by 0
  12. 2026Reinforcement Learning on the Discrete Composition Channel of a Crystal Generator: Validated Gains and Reward Hackingcited by 0
1 more
  1. 2026DiffGA: Steering Frozen Diffusion Models for Materials Design via Evolutionary Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PLaID++: A Preference Aligned Language Model for Targeted Inorganic Materials Design
  2. 2025CrystalGym: A New Benchmark for Materials Discovery Using Reinforcement Learning
  3. 2026Towards Automated Discovery: A Review of Generative Models, Multimodal Learning and Closed-Loop Workflows in Inverse Materials Design
  4. 2025AI-driven materials design: a mini-review
  5. 2026Open Materials Generation with Inference-Time Reinforcement Learning
  6. 2025AIMatDesign: Knowledge-Augmented Reinforcement Learning for Inverse Materials Design under Data Scarcity
  7. 2026Chemical filters for ultra-high-throughput materials screening and generation
  8. 2026CrystalReasoner: Reasoning and RL for Property-Conditioned Crystal Structure Generation