# LLM-Prop: Predicting Physical And Electronic Properties Of Crystalline Solids From Their Text Descriptions

arXiv: https://arxiv.org/abs/2310.14029 · HTML: https://mlsanity.com/paper/2310.14029/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 21 Oct 2023.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Do NLP Models Know Numbers? Probing Numeracy in Embeddings](https://mlsanity.com/paper/1909.07940/) (2019)
- [Injecting Numerical Reasoning Skills into Language Models](https://mlsanity.com/paper/2004.04487/) (2020)
- [Periodic Graph Transformers for Crystal Material Property Prediction](https://mlsanity.com/paper/2209.11807/) (2022)
- [Representing Numbers in NLP: a Survey and a Vision](https://mlsanity.com/paper/2103.13136/) (2021)
- [MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction](https://mlsanity.com/paper/2109.15290/) (2021)
- [Advanced Graph and Sequence Neural Networks for Molecular Property Prediction and Drug Discovery](https://mlsanity.com/paper/2012.01981/) (2020)
- [MatSci-NLP: Evaluating Scientific Language Models on Materials Science Language Tasks Using Text-to-Schema Modeling](https://mlsanity.com/paper/2305.08264/) (2023)
- [KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundi](https://mlsanity.com/paper/2010.12174/) (2020)
- [Equivariant Networks for Crystal Structures](https://mlsanity.com/paper/2211.15420/) (2022)
- [Leveraging Language Representation for Material Recommendation, Ranking, and Exploration](https://mlsanity.com/paper/2305.01101/) (2023)

## Built on by (most cited first, 29 of 29)

- [A Survey on Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2402.13116/) (2024)
- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery](https://mlsanity.com/paper/2406.10833/) (2024)
- [Space Group Informed Transformer for Crystalline Materials Generation](https://mlsanity.com/paper/2403.15734/) (2024)
- [Are LLMs Ready for Real-World Materials Discovery?](https://mlsanity.com/paper/2402.05200/) (2024)
- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [Graph Machine Learning in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2404.14928/) (2024)
- [Materials science in the era of large language models: a perspective](https://mlsanity.com/paper/2403.06949/) (2024)
- [MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding](https://mlsanity.com/paper/2407.04903/) (2024)
- [LLM4Mat-Bench: Benchmarking Large Language Models for Materials Property Prediction](https://mlsanity.com/paper/2411.00177/) (2024)
- [Multimodal Foundation Models for Material Property Prediction and Discovery](https://mlsanity.com/paper/2312.00111/) (2023)
- [ChIRAAG: ChatGPT Informed Rapid and Automated Assertion Generation](https://mlsanity.com/paper/2402.00093/) (2024)
- [A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs](https://mlsanity.com/paper/2410.18779/) (2024)
- [ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domain](https://mlsanity.com/paper/2411.16736/) (2024)
- [Less can be more for predicting properties with large language models](https://mlsanity.com/paper/2406.17295/) (2024)
- [34 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discovery](https://mlsanity.com/paper/2505.03049/) (2025)
- [ChemPile: A 250GB Diverse and Curated Dataset for Chemical Foundation Models](https://mlsanity.com/paper/2505.12534/) (2025)
- [Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on Materials Characterization](https://mlsanity.com/paper/2509.09307/) (2025)
- [Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry](https://mlsanity.com/paper/2411.15221/) (2024)
- [Bridging Text and Molecule: A Survey on Multimodal Frameworks for Molecule](https://mlsanity.com/paper/2403.13830/) (2024)
- [Towards Agentic Intelligence for Materials Science](https://mlsanity.com/paper/2602.00169/) (2026)
- [Reflectance Estimation for Proximity Sensing by Vision-Language Models: Utilizing Distributional Semantics for Low-Level Cognition in Robotics](https://mlsanity.com/paper/2404.07717/) (2024)
- [AGAPI-Agents: An Open-Access Agentic AI Platform for Accelerated Materials Design on AtomGPT.org](https://mlsanity.com/paper/2512.11935/) (2025)
- [SiPhy: Single-Image Physical Property Reasoning](https://mlsanity.com/paper/2607.22355/) (2026)
- [Proof-Carrying Materials: Falsifiable Safety Certificates for Machine-Learned Interatomic Potentials](https://mlsanity.com/paper/2603.12183/) (2026)
- [Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model](https://mlsanity.com/paper/2607.20058/) (2026)
- [Sculpting Molecules in Text-3D Space: A Flexible Substructure Aware Framework for Text-Oriented Molecular Optimization](https://mlsanity.com/paper/2403.03425/) (2024)
- [Towards Efficient Large Language Models for Scientific Text: A Review](https://mlsanity.com/paper/2408.10729/) (2024)
- [OPENXRD: A Comprehensive Benchmark Framework for LLM/MLLM XRD Question Answering](https://mlsanity.com/paper/2507.09155/) (2025)

## Abstract (arXiv)

The prediction of crystal properties plays a crucial role in the crystal design process. Current methods for predicting crystal properties focus on modeling crystal structures using graph neural networks (GNNs). Although GNNs are powerful, accurately modeling the complex interactions between atoms and molecules within a crystal remains a challenge. Surprisingly, predicting crystal properties from crystal text descriptions is understudied, despite the rich information and expressiveness that text data offer. One of the main reasons is the lack of publicly available data for this task. In this paper, we develop and make public a benchmark dataset (called TextEdge) that contains text descriptions of crystal structures with their properties. We then propose LLM-Prop, a method that leverages the general-purpose learning capabilities of large language models (LLMs) to predict the physical and electronic properties of crystals from their text descriptions. LLM-Prop outperforms the current state-of-the-art GNN-based crystal property predictor by about 4% in predicting band gap, 3% in classifying whether the band gap is direct or indirect, and 66% in predicting unit cell volume. LLM-Prop also outperforms a finetuned MatBERT, a domain-specific pre-trained BERT model, despite having 3 times fewer parameters. Our empirical results may highlight the current inability of GNNs to capture information pertaining to space group symmetry and Wyckoff sites for accurate crystal property prediction.