# Predicting New Research Directions in Materials Science using Large Language Models and Concept Graphs

arXiv: https://arxiv.org/abs/2506.16824 · HTML: https://mlsanity.com/paper/2506.16824/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 63 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models](https://mlsanity.com/paper/2404.07738/) (2024)
- [SciMON: Scientific Inspiration Machines Optimized for Novelty](https://mlsanity.com/paper/2305.14259/) (2023)
- [OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts](https://mlsanity.com/paper/2205.01833/) (2022)
- [MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction](https://mlsanity.com/paper/2109.15290/) (2021)
- [Structured information extraction from complex scientific text with fine-tuned large language models](https://mlsanity.com/paper/2212.05238/) (2022)
- [Natural Language Processing Advancements By Deep Learning: A Survey](https://mlsanity.com/paper/2003.01200/) (2020)
- [Predicting the Future of AI with AI: High-quality link prediction in an exponentially growing knowledge network](https://mlsanity.com/paper/2210.00881/) (2022)
- [Predicting Research Trends with Semantic and Neural Networks with an application in Quantum Physics](https://mlsanity.com/paper/1906.06843/) (2019)
- [Generating Interesting Scientific Ideas using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaders](https://mlsanity.com/paper/2405.17044/) (2024)
- [In-situ graph reasoning and knowledge expansion using Graph-PReFLexOR](https://mlsanity.com/paper/2501.08120/) (2025)
- [SCI-IDEA: Context-Aware Scientific Ideation Using Token and Sentence Embeddings](https://mlsanity.com/paper/2503.19257/) (2025)
- [A Big Data Approach to Computational Creativity](https://mlsanity.com/paper/1311.1213/) (2013)
- [ConExion: Concept Extraction with Large Language Models](https://mlsanity.com/paper/2504.12915/) (2025)

## Built on by (most cited first, 9 of 9)

- [Towards autonomous quantum physics research using LLM agents with access to intelligent tools](https://mlsanity.com/paper/2511.11752/) (2025)
- [Harnessing AtomisticSkills for Agentic Atomistic Research](https://mlsanity.com/paper/2605.24002/) (2026)
- [Evidence-Grounded Frontier Mapping and Agentic Hypothesis Generation in Nanomedicine](https://mlsanity.com/paper/2605.18144/) (2026)
- [SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery](https://mlsanity.com/paper/2607.29347/) (2026)
- [HyGRAIL: Cost-Aware and Evidence-Grounded Scientific Hypothesis Discovery over Knowledge Graphs](https://mlsanity.com/paper/2609.02056/) (2026)
- [RAP: Research Attention Prediction Reveals Target-Conditioned Evidence Acquisition Biases](https://mlsanity.com/paper/2609.10092/) (2026)
- [Time-Aligned Evolving Concept Graphs for Scientific Relation Forecasting](https://mlsanity.com/paper/2609.18163/) (2026)
- [SCoR: A Hierarchical Framework for Forecasting Relations Between Scientific Concepts](https://mlsanity.com/paper/2609.22174/) (2026)
- [LANTERN: Illuminating Hidden Mathematical Knowledge in Language Models](https://mlsanity.com/paper/2609.32264/) (2026)

## Abstract (arXiv)

Due to an exponential increase in published research articles, it is impossible for individual scientists to read all publications, even within their own research field. In this work, we investigate the use of large language models (LLMs) for the purpose of extracting the main concepts and semantic information from scientific abstracts in the domain of materials science to find links that were not noticed by humans and thus to suggest inspiring near/mid-term future research directions. We show that LLMs can extract concepts more efficiently than automated keyword extraction methods to build a concept graph as an abstraction of the scientific literature. A machine learning model is trained to predict emerging combinations of concepts, i.e. new research ideas, based on historical data. We demonstrate that integrating semantic concept information leads to an increased prediction performance. The applicability of our model is demonstrated in qualitative interviews with domain experts based on individualized model suggestions. We show that the model can inspire materials scientists in their creative thinking process by predicting innovative combinations of topics that have not yet been investigated.