mlsanityData to 14 Oct 2026

Towards an automated workflow in materials science for combining multi-modal simulative and experimental information using data mining and large language models

arXiv 2502.14904 ↗First posted 18 Feb 2025cond-mat.mtrl-sci cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

To retrieve and compare scientific data of simulations and experiments in materials science, data needs to be easily accessible and machine readable to qualify and quantify various materials science phenomena. The recent progress in open science leverages the accessibility to data. However, a majority of information is encoded within scientific documents limiting the capability of finding suitable literature as well as material properties. This manuscript showcases an automated workflow, which unravels the encoded information from scientific literature to a machine readable data structure of texts, figures, tables, equations and meta-data, using natural language processing and language as well as vision transformer models to generate a machine-readable database. The machine-readable database can be enriched with local data, as e.g. unpublished or private material data, leading to knowledge synthesis. The study shows that such an automated workflow accelerates information retrieval, proximate context detection and material property extraction from multi-modal input data exemplarily shown for the research field of microstructural analyses of face-centered cubic single crystals. Ultimately, a Retrieval-Augmented Generation (RAG) based Large Language Model (LLM) enables a fast and efficient question answering chat bot.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 20 of its 53 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

20 of 53 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  4. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  5. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
  6. 2023Extracting Accurate Materials Data from Research Papers with Conversational Language Models and Prompt Engineeringcited by 86
  7. 202314 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathoncited by 86
  8. 2024Searching for Best Practices in Retrieval-Augmented Generationcited by 83
  9. 2024ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learningcited by 64
  10. 2023Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directionscited by 55
  11. 2023From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Designcited by 51
  12. 2023MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalitiescited by 38
8 more
  1. 2022Physics Guided Deep Learning for Generative Design of Crystal Materials with Symmetry Constraintscited by 30
  2. 2023Opportunities for Retrieval and Tool Augmented Large Language Models in Scientific Facilitiescited by 27
  3. 2024Accelerating Scientific Discovery with Generative Knowledge Extraction, Graph-Based Representation, and Multimodal Intelligent Graph Reasoningcited by 27
  4. 2024Materials science in the era of large language models: a perspectivecited by 24
  5. 2024AMGPT: a Large Language Model for Contextual Querying in Additive Manufacturingcited by 23
  6. 2022Accelerating Material Design with the Generative Toolkit for Scientific Discoverycited by 16
  7. 2024Mining experimental data from Materials Science literature with Large Language Models: an evaluation studycited by 11
  8. 2024Cephalo: Multi-Modal Vision-Language Models for Bio-Inspired Materials Analysis and Designcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Ontology-aligned structuring and reuse of multimodal materials data and workflows towards automatic reproductioncited by 2
  2. 2025Zero-shot reasoning for simulating scholarly peer-reviewcited by 0
  3. 2026Agentic Resource Allocation for Batch Multi-Objective Bayesian Optimization in Autonomous Materials Discoverycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Role of Large Language Models and Retrieval-Augmented Generation for Accelerating Crystalline Material Discovery: A Systematic Review
  2. 2024Are LLMs Ready for Real-World Materials Discovery?
  3. 2024Foundational Large Language Models for Materials Research
  4. 2025Large language models in materials science and the need for open-source approaches
  5. 2026MaterialFigBENCH: benchmark dataset with figures for evaluating college-level materials science problem-solving abilities of multimodal large language models
  6. 2024LLaMP: Large Language Model Made Powerful for High-fidelity Materials Knowledge Retrieval and Distillation
  7. 2025MatQnA: A Benchmark Dataset for Multi-modal Large Language Models in Materials Characterization and Analysis
  8. 202534 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discovery