mlsanityData to 7 Oct 2026

EAA: Automating materials characterization with vision language model agents

arXiv 2602.15294 ↗First posted 17 Feb 2026cs.AIcited by 1

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

We present Experiment Automation Agents (EAA), a vision-language-model-driven agentic system designed to automate complex experimental microscopy workflows. EAA integrates multimodal reasoning, tool-augmented action, and optional long-term memory to support both autonomous procedures and interactive user-guided measurements. Built on a flexible task-manager architecture, the system enables workflows ranging from fully agent-driven automation to logic-defined routines that embed localized LLM queries. EAA further provides a modern tool ecosystem with two-way compatibility for Model Context Protocol (MCP), allowing instrument-control tools to be consumed or served across applications. We demonstrate EAA at an imaging beamline at the Advanced Photon Source, including automated zone plate focusing, natural language-described feature search, and interactive data acquisition. These results illustrate how vision-capable agents can enhance beamline efficiency, reduce operational burden, and lower the expertise barrier for users.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 6 of its 14 references that are in the map. First posted on arXiv 17 Feb 2026.

Builds on

6 of 14 references.

References · 6

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2023Opportunities for Retrieval and Tool Augmented Large Language Models in Scientific Facilitiescited by 27
  4. 2025Collaborating with AI Agents: Field Experiments on Teamwork, Productivity, and Performancecited by 14
  5. 2024VISION: A Modular AI Assistant for Natural Human-Instrument Interaction at Scientific User Facilitiescited by 10
  6. 2025Demonstration of an AI-driven workflow for dynamic x-ray spectroscopycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Physics as the label for measuring and correcting materials reasoning in multimodal modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Autonomous Microscopy Experiments through Large Language Model Agents
  2. 2026Agentic self-driving microscopy benchmarks support qualification but do not necessarily generalize to unseen tasks
  3. 2026Agentic AI for operating scientific instruments for nanoscale characterization
  4. 2024PEAR: A Robust and Flexible Automation Framework for Ptychography Enabled by Multiple Large Language Model Agents
  5. 2026OpenQlaw: An Agentic AI Assistant for Analysis of 2D Quantum Materials
  6. 2025Toward Greater Autonomy in Materials Discovery Agents: Unifying Planning, Physics, and Scientists
  7. 2025AutoMat: Enabling Automated Crystal Structure Reconstruction from Microscopy via Agentic Tool Use
  8. 2026AutoXRD: Autonomous LLM Agents and Comprehensive Evaluation for Powder Diffraction Analysis