mlsanityData to 7 Oct 2026

Materials science in the era of large language models: a perspective

arXiv 2403.06949 ↗First posted 11 Mar 2024cond-mat.mtrl-sci cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have garnered considerable interest due to their impressive natural language capabilities, which in conjunction with various emergent properties make them versatile tools in workflows ranging from complex code generation to heuristic finding for combinatorial problems. In this paper we offer a perspective on their applicability to materials science research, arguing their ability to handle ambiguous requirements across a range of tasks and disciplines mean they could be a powerful tool to aid researchers. We qualitatively examine basic LLM theory, connecting it to relevant properties and techniques in the literature before providing two case studies that demonstrate their use in task automation and knowledge extraction at-scale. At their current stage of development, we argue LLMs should be viewed less as oracles of novel insight, and more as tireless workers that can accelerate and unify exploration across domains. It is our hope that this paper can familiarise material science researchers with the concepts needed to leverage these tools in their own research.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 93 of its 146 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

93 of 146 references.

References · 93

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2017Proximal Policy Optimization Algorithmscited by 14,441
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  11. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  12. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
38 more
  1. 2023Segment Anythingcited by 8,158
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2017Deep reinforcement learning from human preferencescited by 3,846
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  12. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  13. 2020Longformer: The Long-Document Transformercited by 3,362
  14. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  15. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  16. 2022Emergent Abilities of Large Language Modelscited by 2,483
  17. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  18. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  19. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  20. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  21. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  22. 2019Language Models as Knowledge Bases?cited by 1,922
  23. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  24. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  25. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  26. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  27. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  28. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  29. 2023Large Language Models as Optimizerscited by 830
  30. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  31. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  32. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  33. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  34. 2022Large Language Models Can Self-Improvecited by 656
  35. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  36. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  37. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  38. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 8; 2026: 9.

23 papers.

Sort:

By area:

  1. 2024Generative Hierarchical Materials Searchcited by 12
  2. 2024AI-driven inverse design of materials: Past, present and futurecited by 10
  3. 2024Evaluating the Performance and Robustness of LLMs in Materials Science Q&A and Property Predictionscited by 9
  4. 2024MetaScientist: A Human-AI Synergistic Framework for Automated Mechanical Metamaterial Designcited by 8
  5. 2025Zephyrus: An Agentic Framework for Weather Sciencecited by 8
  6. 2024A Survey of Automatic Hallucination Evaluation on Natural Language Generationcited by 7
  7. 2024AstroMLab 2: AstroLLaMA-2-70B Model and Benchmarking Specialised LLMs for Astronomycited by 6
  8. 2025Generative AI Uses and Risks for Knowledge Workers in a Science Organizationcited by 5
  9. 2026SimMOF: AI agent for Automated MOF Simulationscited by 4
  10. 2025MolEdit: Knowledge Editing for Multimodal Molecule Language Modelscited by 3
  11. 2025Towards an automated workflow in materials science for combining multi-modal simulative and experimental information using data mining and large language modelscited by 2
  12. 2025Towards Fully-Automated Materials Discovery via Large-Scale Synthesis Dataset and Expert-Level LLM-as-a-Judgecited by 2
11 more
  1. 2025aLLoyM: A large language model for alloy phase diagram predictioncited by 1
  2. 2026TurboAgent: An LLM-Driven Autonomous Multi-Agent Framework for Turbomachinery Aerodynamic Designcited by 1
  3. 2025Linking heterogeneous microstructure informatics with expert characterization knowledge through customized and hybrid vision-language representations for industrial qualificationcited by 0
  4. 2025Accelerating Materials Discovery: Learning a Universal Representation of Chemical Processes for Cross-Domain Property Predictioncited by 0
  5. 2026Sustainable Materials Discovery in the Era of Artificial Intelligencecited by 0
  6. 2026From Word2Vec to Transformers: Text-Derived Composition Embeddings for Filtering Combinatorial Electrocatalystscited by 0
  7. 2026Intelligent Materials Modelling: Large Language Models Versus Partial Least Squares Regression for Predicting Polysulfone Membrane Mechanical Performancecited by 0
  8. 2026A collaborative agent with two lightweight synergistic models for autonomous crystal materials researchcited by 0
  9. 2026Interpretable Inverse Design of Metal-Organic Frameworks with Large Language Model Agentscited by 0
  10. 2026Skill-Contracted Agents for Evidence-Aware Materials Literature Analysiscited by 0
  11. 2026Enhancing EBSD throughput of battery electrode materials using super-resolution generative adversarial networkscited by 0