# Regression with Large Language Models for Materials and Molecular Property Prediction

arXiv: https://arxiv.org/abs/2409.06080 · HTML: https://mlsanity.com/paper/2409.06080/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 49 references that are in the map.
- First posted on arXiv 9 Sep 2024.

## Builds on (most cited first, 14 of 14)

- [Large-Scale Chemical Language Representations Capture Molecular Structure and Properties](https://mlsanity.com/paper/2106.09553/) (2021)
- [Translation between Molecules and Natural Language](https://mlsanity.com/paper/2204.11817/) (2022)
- [LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset](https://mlsanity.com/paper/2402.09391/) (2024)
- [ChemBERTa-2: Towards Chemical Foundation Models](https://mlsanity.com/paper/2209.01712/) (2022)
- [Extracting Accurate Materials Data from Research Papers with Conversational Language Models and Prompt Engineering](https://mlsanity.com/paper/2303.05352/) (2023)
- [InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery](https://mlsanity.com/paper/2311.16208/) (2023)
- [Predicting materials properties without crystal structure: Deep representation learning from stoichiometry](https://mlsanity.com/paper/1910.00617/) (2019)
- [BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning](https://mlsanity.com/paper/2402.17810/) (2024)
- [Machine learning modeling of superconducting critical temperature](https://mlsanity.com/paper/1709.02727/) (2017)
- [ReLM: Leveraging Language Models for Enhanced Chemical Reaction Prediction](https://mlsanity.com/paper/2310.13590/) (2023)
- [BARTSmiles: Generative Masked Language Models for Molecular Representations](https://mlsanity.com/paper/2211.16349/) (2022)
- [Multimodal Language and Graph Learning of Adsorption Configuration in Catalysis](https://mlsanity.com/paper/2401.07408/) (2024)
- [A Universal Framework for Accurate and Efficient Geometric Deep Learning of Molecular Systems](https://mlsanity.com/paper/2311.11228/) (2023)
- [Can Large Language Models Understand Molecules?](https://mlsanity.com/paper/2402.00024/) (2024)

## Built on by (most cited first, 11 of 11)

- [34 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discovery](https://mlsanity.com/paper/2505.03049/) (2025)
- [Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry](https://mlsanity.com/paper/2411.15221/) (2024)
- [BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models](https://mlsanity.com/paper/2505.01912/) (2025)
- [SiPhy: Single-Image Physical Property Reasoning](https://mlsanity.com/paper/2607.22355/) (2026)
- [Distilling and exploiting quantitative insights from Large Language Models for enhanced Bayesian optimization of chemical reactions](https://mlsanity.com/paper/2504.08874/) (2025)
- [Text to Band Gap: Pre-trained Language Models as Encoders for Semiconductor Band Gap Prediction](https://mlsanity.com/paper/2501.03456/) (2025)
- [Post Hoc Regression Refinement via Pairwise Rankings](https://mlsanity.com/paper/2508.16495/) (2025)
- [Element2Vec: Build Chemical Element Representation from Text for Property Prediction](https://mlsanity.com/paper/2510.13916/) (2025)
- [Rethinking Molecular Text Representations for LLMs: An Empirical Study](https://mlsanity.com/paper/2606.03057/) (2026)
- [From Critic to Confidence: PPO for Language-Based Quantitative Prediction with Confidence Estimation](https://mlsanity.com/paper/2607.12687/) (2026)
- [LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators](https://mlsanity.com/paper/2610.17246/) (2026)

## Abstract (arXiv)

We demonstrate the ability of large language models (LLMs) to perform material and molecular property regression tasks, a significant deviation from the conventional LLM use case. We benchmark the Large Language Model Meta AI (LLaMA) 3 on several molecular properties in the QM9 dataset and 28 materials properties. Only composition-based input strings are used as the model input and we fine tune on only the generative loss. We broadly find that LLaMA 3, when fine-tuned using the SMILES representation of molecules, provides useful regression results which can rival standard materials property prediction models like random forest or fully connected neural networks on the QM9 dataset. Not surprisingly, LLaMA 3 errors are 5-10x higher than those of the state-of-the-art models that were trained using far more granular representation of molecules (e.g., atom types and their coordinates) for the same task. Similarly, LLaMA 3 provides comparable, although slightly worse, accuracy relative to random forest and elemental descriptors when using just compound chemical description on our set of 28 materials properties. Interestingly, LLaMA 3 provides improved predictions compared to GPT-3.5 and GPT-4o. This work highlights the versatility of LLMs, suggesting that LLM-like generative models can potentially transcend their traditional applications to tackle complex physical phenomena, thus paving the way for future research and applications in chemistry, materials science and other scientific domains.