mlsanityData to 7 Oct 2026

FlowLLM: Flow Matching for Material Generation with Large Language Models as Base Distributions

arXiv 2410.23405 ↗First posted 30 Oct 2024cs.LG cond-mat.mtrl-sci cs.AIcited by 52

On the map: Diffusion models → Flow matching, time series and science

Abstract (arXiv)

Material discovery is a critical area of research with the potential to revolutionize various fields, including carbon capture, renewable energy, and electronics. However, the immense scale of the chemical space makes it challenging to explore all possible materials experimentally. In this paper, we introduce FlowLLM, a novel generative model that combines large language models (LLMs) and Riemannian flow matching (RFM) to design novel crystalline materials. FlowLLM first fine-tunes an LLM to learn an effective base distribution of meta-stable crystals in a text representation. After converting to a graph representation, the RFM model takes samples from the LLM and iteratively refines the coordinates and lattice parameters. Our approach significantly outperforms state-of-the-art methods, increasing the generation rate of stable materials by over three times and increasing the rate for stable, unique, and novel crystals by ~50% - a huge improvement on a difficult problem. Additionally, the crystals generated by FlowLLM are much closer to their relaxed state when compared with another leading model, significantly reducing post-hoc computational cost.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 31 of its 52 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

31 of 52 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Diffusion probabilistic models enhance variational autoencoder for crystal structure generative modeling4 Aug
  2. 2023Crystal Structure Prediction by Joint Equivariant Diffusion30 Jul
  3. 2022Flow Matching for Generative Modeling6 Oct

Other references · 28

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2021E(n) Equivariant Graph Neural Networkscited by 951
  7. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  8. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  9. 2021Crystal Diffusion Variational Autoencoder for Periodic Material Generationcited by 268
  10. 2020Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densitiescited by 248
  11. 2020The Open Catalyst 2020 (OC20) Dataset and Community Challengescited by 246
  12. 2023EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representationscited by 201
16 more
  1. 2022e3nn: Euclidean Neural Networkscited by 171
  2. 2023Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNscited by 113
  3. 2024FlowMM: Generating Materials with Riemannian Flow Matchingcited by 105
  4. 2023MatterGen: a generative model for inorganic materials designcited by 98
  5. 2024Fine-Tuned Language Models Generate Stable Inorganic Materials as Textcited by 92
  6. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  7. 2021Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifoldscited by 72
  8. 2024Space Group Constrained Crystal Generationcited by 71
  9. 2023Language models can generate molecules, materials, and protein binding sites directly in three dimensions as XYZ, CIF, and PDB filescited by 59
  10. 2020An invertible crystallographic representation for general inverse design of inorganic crystals with targeted propertiescited by 53
  11. 2023Scalable Diffusion for Materials Generationcited by 52
  12. 2020Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Propertiescited by 51
  13. 2022Spherical Channels for Modeling Atomic Interactionscited by 47
  14. 2021Normalizing flows for atomic solidscited by 39
  15. 2023The Open DAC 2023 Dataset and Challenges for Sorbent Discovery in Direct Air Capturecited by 18
  16. 2022Large-scale machine-learning-assisted exploration of the whole materials spacecited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 31; 2026: 20.

52 papers.

Sort:

By area:

  1. 2025All-atom Diffusion Transformers: Unified generative modelling of molecules and materialscited by 75
  2. 2025Open Materials Generation with Stochastic Interpolantscited by 26
  3. 2025LeMat-GenBench: A Unified Evaluation Framework for Crystal Generative Modelscited by 20
  4. 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
  5. 2025Space Group Equivariant Crystal Diffusioncited by 15
  6. 2025LLM Meets Diffusion: A Hybrid Framework for Crystal Material Generationcited by 12
  7. 2025PLaID++: A Preference Aligned Language Model for Targeted Inorganic Materials Designcited by 11
  8. 2024AI-driven inverse design of materials: Past, present and futurecited by 10
  9. 2025Toward Greater Autonomy in Materials Discovery Agents: Unifying Planning, Physics, and Scientistscited by 10
  10. 2025All that structure matches does not glittercited by 10
  11. 2025Symmetry-Aware Bayesian Flow Networks for Crystal Generationcited by 9
  12. 2025Materials Generation in the Era of Artificial Intelligence: A Comprehensive Surveycited by 9
38 more
  1. 2025Kinetic Langevin Diffusion for Crystalline Materials Generationcited by 9
  2. 2026Crystalite: A Lightweight Transformer for Efficient Crystal Modelingcited by 9
  3. 2025Establishing baselines for generative discovery of inorganic crystalscited by 8
  4. 2025Artificial Intelligence and Generative Models for Materials Discovery -- A Reviewcited by 8
  5. 2025MatLLMSearch: Crystal Structure Discovery with Evolution-Guided Large Language Modelscited by 7
  6. 2025A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Toolscited by 7
  7. 2025Space Group Conditional Flow Matchingcited by 7
  8. 2025Efficient Flow Matching using Latent Variablescited by 6
  9. 2025Language Models Enable Data-Augmented Synthesis Planning for Inorganic Materialscited by 6
  10. 2025TopoMAS: Large Language Model Driven Topological Materials Multiagent Systemcited by 6
  11. 2025CrystalDiT: A Diffusion Transformer for Crystal Generationcited by 6
  12. 2026Open Materials Generation with Inference-Time Reinforcement Learningcited by 6
  13. 2025CrystalGRW: Generative Modeling of Crystal Structures with Targeted Properties via Geodesic Random Walkscited by 5
  14. 2025Continuous SUN (Stable, Unique, and Novel) Metric for Generative Modeling of Inorganic Crystalscited by 5
  15. 2025Boltzmann generators for amorphous particle systemscited by 5
  16. 2026Multimodal Crystal Flow: Any-to-Any Modality Generation for Unified Crystal Modelingcited by 4
  17. 2025MiAD: Mirage Atom Diffusion for De Novo Crystal Generationcited by 3
  18. 2025Generative Models for Crystalline Materialscited by 3
  19. 2025PhononBench:A Large-Scale Phonon-Based Benchmark for Dynamical Stability in Crystal Generationcited by 3
  20. 2026Zatom-1: Towards a Multimodal Foundation Model for 3D Molecules and Materialscited by 3
  21. 2026SLayerGen: a Crystal Generative Model for all Space and Layer Groupscited by 2
  22. 2026CrystalREPA: Transferring Physical Priors from Universal MLIPs to Crystal Generative Modelscited by 2
  23. 2025CRYSIM: Prediction of Symmetric Structures of Large Crystals with GPU-based Ising Machinescited by 1
  24. 2025Perovskite-R1: a domain-specialized large language model for intelligent discovery of precursor additives and experimental designcited by 1
  25. 2025Preference Learning from Physics-Based Feedback: Tuning Language Models to Design BCC/B2 Superalloyscited by 1
  26. 2026C2NP: A Benchmark for Learning Scale-Dependent Geometric Invariances in 3D Materials Generationcited by 1
  27. 2026Crys-JEPA: Accelerating Crystal Discovery via Embedding Screening and Generative Refinementcited by 1
  28. 2026Composable Crystals: Controllable Materials Discovery via Concept Learningcited by 1
  29. 2026SinAE: A Single-Architecture Flow-Matching Autoencoder for Cross-Domain Atomic Systemscited by 1
  30. 2025Decision Flow Policy Optimizationcited by 0
  31. 2026SCALAR: Quantifying Structural Hallucination, Consistency, and Reasoning Gaps in Materials Foundation Modelscited by 0
  32. 2026OrgFlow: Generative Modeling of Organic Crystal Structures from Molecular Graphscited by 0
  33. 2026Lang2Str: Two-Stage Crystal Structure Generation with LLMs and Continuous Flow Modelscited by 0
  34. 2026AMGenC: Generating Charge Balanced Amorphous Materialscited by 0
  35. 2026Sharpen Your Flow: Sharpness-Aware Sampling for Flow Matchingcited by 0
  36. 2026PRISMat: Policy-Driven, Permutation-Invariant Autoregressive Material Generationcited by 0
  37. 2026Learning Unbiased Permutations via Flow Matchingcited by 0
  38. 2026Representations from Pretrained Machine-Learning Interatomic Potentials as Coarse Coordinates for Material Generation and Evaluationcited by 0

50 most cited of 52 All 52 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026DMFlow: Disordered Materials Generation by Flow Matching
  2. 2025Role of Large Language Models and Retrieval-Augmented Generation for Accelerating Crystalline Material Discovery: A Systematic Review
  3. 2026MatMind: A Structure-Activity Knowledge-Driven Generative Foundation Model for Materials Science
  4. 2024LLMatDesign: Autonomous Materials Discovery with Large Language Models
  5. 2026OMatG-flash: An All-Atom Flow Map with Reinforce Adjoint Matching for Scalable Materials Discovery
  6. 2025Large language models in materials science and the need for open-source approaches
  7. 2024Generative Hierarchical Materials Search
  8. 2024Unleashing the power of novel conditional generative approaches for new materials discovery