mlsanityData to 7 Oct 2026

Are LLMs Ready for Real-World Materials Discovery?

arXiv 2402.05200 ↗First posted 7 Feb 2024cond-mat.mtrl-sci cs.AI cs.CLcited by 40

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) create exciting possibilities for powerful language processing tools to accelerate research in materials science. While LLMs have great potential to accelerate materials understanding and discovery, they currently fall short in being practical materials science tools. In this position paper, we show relevant failure cases of LLMs in materials science that reveal current limitations of LLMs related to comprehending and reasoning over complex, interconnected materials science knowledge. Given those shortcomings, we outline a framework for developing Materials Science LLMs (MatSci-LLMs) that are grounded in materials science knowledge and hypothesis generation followed by hypothesis testing. The path to attaining performant MatSci-LLMs rests in large part on building high-quality, multi-modal datasets sourced from scientific literature where various information extraction challenges persist. As such, we describe key materials science information extraction challenges which need to be overcome in order to build large-scale, multi-modal datasets that capture valuable materials science knowledge. Finally, we outline a roadmap for applying future MatSci-LLMs for real-world materials discovery via: 1. Automated Knowledge Base Generation; 2. Automated In-Silico Material Design; and 3. MatSci-LLM Integrated Self-Driving Materials Laboratories.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 31 of its 83 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

31 of 83 references.

References · 31

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2023BloombergGPT: A Large Language Model for Financecited by 923
  6. 2023Large Language Modelscited by 774
  7. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  8. 2021Crystal Diffusion Variational Autoencoder for Periodic Material Generationcited by 268
  9. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 264
  10. 2023Large Language Models for Software Engineering: Survey and Open Problemscited by 178
  11. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  12. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
19 more
  1. 2023Crystal Structure Prediction by Joint Equivariant Diffusioncited by 142
  2. 2022Large Language Models Meet NL2Code: A Surveycited by 135
  3. 2023MatterGen: a generative model for inorganic materials designcited by 98
  4. 2024Fine-Tuned Language Models Generate Stable Inorganic Materials as Textcited by 92
  5. 202314 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathoncited by 85
  6. 2021MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extractioncited by 82
  7. 2023ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Textscited by 78
  8. 2023Language models can generate molecules, materials, and protein binding sites directly in three dimensions as XYZ, CIF, and PDB filescited by 59
  9. 2019The Materials Science Procedural Text Corpus: Annotating Materials Synthesis Procedures with Shallow Semantic Structurescited by 48
  10. 2023DARWIN Series: Domain Specific Large Language Models for Natural Sciencecited by 39
  11. 2023MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalitiescited by 38
  12. 2023MatSci-NLP: Evaluating Scientific Language Models on Materials Science Language Tasks Using Text-to-Schema Modelingcited by 35
  13. 2022Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Modelcited by 32
  14. 2023LLM-Prop: Predicting Physical And Electronic Properties Of Crystalline Solids From Their Text Descriptionscited by 29
  15. 2023HoneyBee: Progressive Instruction Finetuning of Large Language Models for Materials Sciencecited by 22
  16. 2018Inorganic Materials Synthesis Planning with Literature-Trained Neural Networkscited by 20
  17. 2022DiSCoMaT: Distantly Supervised Composition Extraction from Tables in Materials Science Articlescited by 17
  18. 2021Looking Through Glass: Knowledge Discovery from Materials Science Literature using Natural Language Processingcited by 16
  19. 2023Reconstructing Materials Tetrahedron: Challenges in Materials Information Extractioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 11; 2026: 8.

40 papers.

Sort:

By area:

  1. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  2. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  3. 2024A Survey on Self-Evolution of Large Language Modelscited by 86
  4. 2024Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent Systemcited by 63
  5. 2024HoneyComb: A Flexible LLM-Based Agent System for Materials Sciencecited by 56
  6. 2024CiteME: Can Language Models Accurately Cite Scientific Claims?cited by 45
  7. 2024Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challengescited by 41
  8. 2024Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verificationcited by 40
  9. 2024Are large language models superhuman chemists?cited by 34
  10. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  11. 2024Probing the limitations of multimodal language models for chemistry and materials researchcited by 31
  12. 2024MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understandingcited by 23
28 more
  1. 2024MatExpert: Decomposing Materials Discovery by Mimicking Human Expertscited by 18
  2. 2024Foundational Large Language Models for Materials Researchcited by 18
  3. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 14
  4. 2024Generative Adversarial Reviews: When LLMs Become the Criticcited by 12
  5. 2026Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoningcited by 12
  6. 2025LLEMA: Evolutionary Search with LLMs for Multi-Objective Materials Discoverycited by 11
  7. 2024Less can be more for predicting properties with large language modelscited by 10
  8. 2024AI-driven inverse design of materials: Past, present and futurecited by 10
  9. 202534 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discoverycited by 9
  10. 2024MatViX: Multimodal Information Extraction from Visually Rich Articlescited by 8
  11. 2024Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistrycited by 8
  12. 2024MetaScientist: A Human-AI Synergistic Framework for Automated Mechanical Metamaterial Designcited by 8
  13. 2025A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Toolscited by 7
  14. 2026TritonDFT: Automating DFT with a Multi-Agent Frameworkcited by 5
  15. 2025Stress-Testing Multimodal Foundation Models for Crystallographic Reasoningcited by 4
  16. 2025ChemQuests: A Curated Chemistry Question-Answer Database Extracted from ChemRxiv paperscited by 3
  17. 2025MSQA: Benchmarking LLMs on Graduate-Level Materials Science Reasoning and Knowledgecited by 3
  18. 2024Evaluating Large Language Models for Material Selectioncited by 2
  19. 2025Perovskite-LLM: Knowledge-Enhanced Large Language Models for Perovskite Solar Cell Researchcited by 2
  20. 2025CrystalICL: Enabling In-Context Learning for Crystal Generationcited by 2
  21. 2024Towards Foundation Model for Chemical Reactor Modeling: Meta-Learning with Physics-Informed Adaptationcited by 1
  22. 2026Multi-Agent Collaboration for Automated Design Exploration on High Performance Computing Systemscited by 1
  23. 2026Proof-Carrying Materials: Falsifiable Safety Certificates for Machine-Learned Interatomic Potentialscited by 1
  24. 2026From Data to Theory: Autonomous Large Language Model Agents for Materials Sciencecited by 1
  25. 2025Bridging the Gap Between Scientific Laws Derived by AI Systems and Canonical Knowledge via Abductive Inference with AI-Noethercited by 0
  26. 2026Sustainable Materials Discovery in the Era of Artificial Intelligencecited by 0
  27. 2026ArtifactLinker: Linking Scientific Artifacts for Automatic State-of-the-Art Discoverycited by 0
  28. 2026SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Hypothesis Generation for Materials Discovery and Design Using Goal-Driven and Constraint-Guided LLM Agents
  2. 2025Large language models in materials science and the need for open-source approaches
  3. 2025Role of Large Language Models and Retrieval-Augmented Generation for Accelerating Crystalline Material Discovery: A Systematic Review
  4. 2026From Knowledge to Action: Outcomes of the 2025 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry
  5. 2024Mining experimental data from Materials Science literature with Large Language Models: an evaluation study
  6. 2024LLaMP: Large Language Model Made Powerful for High-fidelity Materials Knowledge Retrieval and Distillation
  7. 2024Evaluating the Performance and Robustness of LLMs in Materials Science Q&A and Property Predictions
  8. 2025MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Science