# Opportunities for Retrieval and Tool Augmented Large Language Models in Scientific Facilities

arXiv: https://arxiv.org/abs/2312.01291 · HTML: https://mlsanity.com/paper/2312.01291/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2023.

## Built on by (most cited first, 27 of 27)

- [PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2402.07867/) (2024)
- [A Review of Large Language Models and Autonomous Agents in Chemistry](https://mlsanity.com/paper/2407.01603/) (2024)
- [Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2503.08979/) (2025)
- [Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applications](https://mlsanity.com/paper/2411.14110/) (2024)
- [Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning](https://mlsanity.com/paper/2502.02871/) (2025)
- [One Shot Dominance: Knowledge Poisoning Attack on Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2505.11548/) (2025)
- [Hyper-RAG: Combating LLM Hallucinations using Hypergraph-Driven Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.08758/) (2025)
- [MDCrow: Automating Molecular Dynamics Workflows with Large Language Models](https://mlsanity.com/paper/2502.09565/) (2025)
- [MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation](https://mlsanity.com/paper/2407.01972/) (2024)
- [VISION: A Modular AI Assistant for Natural Human-Instrument Interaction at Scientific User Facilities](https://mlsanity.com/paper/2412.18161/) (2024)
- [From Screens to Scenes: A Survey of Embodied AI in Healthcare](https://mlsanity.com/paper/2501.07468/) (2025)
- [Keyword search is all you need: Achieving RAG-Level Performance without vector databases using agentic tool use](https://mlsanity.com/paper/2602.23368/) (2025)
- [MetaScientist: A Human-AI Synergistic Framework for Automated Mechanical Metamaterial Design](https://mlsanity.com/paper/2412.16270/) (2024)
- [Generative AI Uses and Risks for Knowledge Workers in a Science Organization](https://mlsanity.com/paper/2501.16577/) (2025)
- [CtrlRAG: Black-box Document Poisoning Attacks for Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2503.06950/) (2025)
- [FIRST: Federated Inference Resource Scheduling Toolkit for Scientific AI Model Access](https://mlsanity.com/paper/2510.13724/) (2025)
- [VASP Agent: An Agentic Framework for Autonomous First-principles Calculations](https://mlsanity.com/paper/2512.19458/) (2025)
- [Towards a Science Exocortex](https://mlsanity.com/paper/2406.17809/) (2024)
- [Towards an automated workflow in materials science for combining multi-modal simulative and experimental information using data mining and large language models](https://mlsanity.com/paper/2502.14904/) (2025)
- [Virtual Scientific Companion for Synchrotron Beamlines: A Prototype](https://mlsanity.com/paper/2312.17180/) (2023)
- [GRETEL: A Goal-driven Retrieval and Execution-based Trial Framework for LLM Tool Selection Enhancing](https://mlsanity.com/paper/2510.17843/) (2025)
- [Missing vs. Unused Knowledge Hypothesis for Language Model Bottlenecks in Patent Understanding](https://mlsanity.com/paper/2505.12452/) (2025)
- [Active Inference AI Systems for Scientific Discovery](https://mlsanity.com/paper/2506.21329/) (2025)
- [EAA: Automating materials characterization with vision language model agents](https://mlsanity.com/paper/2602.15294/) (2026)
- [Language-Native Materials Processing Design by Lightly Structured Text Database and Reasoning Large Language Model](https://mlsanity.com/paper/2509.06093/) (2025)
- [HPC-LLM: Practical Domain Adaptation and Retrieval-Augmented Generation for HPC Support](https://mlsanity.com/paper/2605.16347/) (2026)
- [A corrective agentic hybrid RAG and an operations-grounded evaluation for a scientific facility](https://mlsanity.com/paper/2607.24663/) (2026)

## Abstract (arXiv)

Upgrades to advanced scientific user facilities such as next-generation x-ray light sources, nanoscience centers, and neutron facilities are revolutionizing our understanding of materials across the spectrum of the physical sciences, from life sciences to microelectronics. However, these facility and instrument upgrades come with a significant increase in complexity. Driven by more exacting scientific needs, instruments and experiments become more intricate each year. This increased operational complexity makes it ever more challenging for domain scientists to design experiments that effectively leverage the capabilities of and operate on these advanced instruments. Large language models (LLMs) can perform complex information retrieval, assist in knowledge-intensive tasks across applications, and provide guidance on tool usage. Using x-ray light sources, leadership computing, and nanoscience centers as representative examples, we describe preliminary experiments with a Context-Aware Language Model for Science (CALMS) to assist scientists with instrument operations and complex experimentation. With the ability to retrieve relevant information from facility documentation, CALMS can answer simple questions on scientific capabilities and other operational procedures. With the ability to interface with software tools and experimental hardware, CALMS can conversationally operate scientific instruments. By making information more accessible and acting on user needs, LLMs could expand and diversify scientific facilities' users and accelerate scientific output.