# Materials science in the era of large language models: a perspective

arXiv: https://arxiv.org/abs/2403.06949 · HTML: https://mlsanity.com/paper/2403.06949/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 93 of its 146 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 30 of 93)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)

## Built on by (most cited first, 24 of 24)

- [Generative Hierarchical Materials Search](https://mlsanity.com/paper/2409.06762/) (2024)
- [AI-driven inverse design of materials: Past, present and future](https://mlsanity.com/paper/2411.09429/) (2024)
- [Zephyrus: An Agentic Framework for Weather Science](https://mlsanity.com/paper/2510.04017/) (2025)
- [Evaluating the Performance and Robustness of LLMs in Materials Science Q&A and Property Predictions](https://mlsanity.com/paper/2409.14572/) (2024)
- [MetaScientist: A Human-AI Synergistic Framework for Automated Mechanical Metamaterial Design](https://mlsanity.com/paper/2412.16270/) (2024)
- [A Survey of Automatic Hallucination Evaluation on Natural Language Generation](https://mlsanity.com/paper/2404.12041/) (2024)
- [AstroMLab 2: AstroLLaMA-2-70B Model and Benchmarking Specialised LLMs for Astronomy](https://mlsanity.com/paper/2409.19750/) (2024)
- [Generative AI Uses and Risks for Knowledge Workers in a Science Organization](https://mlsanity.com/paper/2501.16577/) (2025)
- [SimMOF: AI agent for Automated MOF Simulations](https://mlsanity.com/paper/2603.29152/) (2026)
- [Towards an automated workflow in materials science for combining multi-modal simulative and experimental information using data mining and large language models](https://mlsanity.com/paper/2502.14904/) (2025)
- [MolEdit: Knowledge Editing for Multimodal Molecule Language Models](https://mlsanity.com/paper/2511.12770/) (2025)
- [Towards Fully-Automated Materials Discovery via Large-Scale Synthesis Dataset and Expert-Level LLM-as-a-Judge](https://mlsanity.com/paper/2502.16457/) (2025)
- [aLLoyM: A large language model for alloy phase diagram prediction](https://mlsanity.com/paper/2507.22558/) (2025)
- [TurboAgent: An LLM-Driven Autonomous Multi-Agent Framework for Turbomachinery Aerodynamic Design](https://mlsanity.com/paper/2604.06747/) (2026)
- [Linking heterogeneous microstructure informatics with expert characterization knowledge through customized and hybrid vision-language representations for industrial qualification](https://mlsanity.com/paper/2508.20243/) (2025)
- [Accelerating Materials Discovery: Learning a Universal Representation of Chemical Processes for Cross-Domain Property Prediction](https://mlsanity.com/paper/2512.05979/) (2025)
- [Sustainable Materials Discovery in the Era of Artificial Intelligence](https://mlsanity.com/paper/2601.21527/) (2026)
- [From Word2Vec to Transformers: Text-Derived Composition Embeddings for Filtering Combinatorial Electrocatalysts](https://mlsanity.com/paper/2603.08881/) (2026)
- [Intelligent Materials Modelling: Large Language Models Versus Partial Least Squares Regression for Predicting Polysulfone Membrane Mechanical Performance](https://mlsanity.com/paper/2603.13834/) (2026)
- [A collaborative agent with two lightweight synergistic models for autonomous crystal materials research](https://mlsanity.com/paper/2604.11540/) (2026)
- [Interpretable Inverse Design of Metal-Organic Frameworks with Large Language Model Agents](https://mlsanity.com/paper/2606.29459/) (2026)
- [Skill-Contracted Agents for Evidence-Aware Materials Literature Analysis](https://mlsanity.com/paper/2607.20431/) (2026)
- [Enhancing EBSD throughput of battery electrode materials using super-resolution generative adversarial networks](https://mlsanity.com/paper/2608.19117/) (2026)
- [Agentic Resource Allocation for Batch Multi-Objective Bayesian Optimization in Autonomous Materials Discovery](https://mlsanity.com/paper/2610.04134/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have garnered considerable interest due to their impressive natural language capabilities, which in conjunction with various emergent properties make them versatile tools in workflows ranging from complex code generation to heuristic finding for combinatorial problems. In this paper we offer a perspective on their applicability to materials science research, arguing their ability to handle ambiguous requirements across a range of tasks and disciplines mean they could be a powerful tool to aid researchers. We qualitatively examine basic LLM theory, connecting it to relevant properties and techniques in the literature before providing two case studies that demonstrate their use in task automation and knowledge extraction at-scale. At their current stage of development, we argue LLMs should be viewed less as oracles of novel insight, and more as tireless workers that can accelerate and unify exploration across domains. It is our hope that this paper can familiarise material science researchers with the concepts needed to leverage these tools in their own research.