mlsanityData to 14 Oct 2026

LAMMI-Pathology: A Tool-Centric Bottom-Up LVLM-Agent Framework for Molecularly Informed Medical Intelligence in Pathology

arXiv 2602.18773 ↗First posted 21 Feb 2026cs.AIcited by 3

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

The emergence of tool-calling-based agent systems introduces a more evidence-driven paradigm for pathology image analysis in contrast to the coarse-grained text-image diagnostic approaches. With the recent large-scale experimental adoption of spatial transcriptomics technologies, molecularly validated pathological diagnosis is becoming increasingly open and accessible. In this work, we propose LAMMI-Pathology (LVLM-Agent System for Molecularly Informed Medical Intelligence in Pathology), a scalable agent framework for domain-specific agent tool-calling. LAMMI-Pathology adopts a tool-centric, bottom-up architecture in which customized domain-adaptive tools serve as the foundation. These tools are clustered by domain style to form component agents, which are then coordinated through a top-level planner hierarchically, avoiding excessively long context lengths that could induce task drift. Based on that, we introduce a novel trajectory construction mechanism based on Atomic Execution Nodes (AENs), which serve as reliable and composable units for building semi-simulated reasoning trajectories that capture credible agent-tool interactions. Building on this foundation, we develop a trajectory-aware fine-tuning strategy that aligns the planner's decision-making process with these multi-step reasoning trajectories, thereby enhancing inference robustness in pathology understanding and its adaptive use of the customized toolset.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 36 of its 62 references that are in the map. First posted on arXiv 21 Feb 2026.

Builds on

36 of 62 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Evidence-based diagnostic reasoning with multi-agent copilot for human pathology26 Jun
  2. 2025PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathology13 Feb

Other references · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  8. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  9. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  10. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  11. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  12. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
22 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  2. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 393
  3. 2024ToolACE: Winning the Points of LLM Function Callingcited by 216
  4. 2023Efficient Guided Generation for Large Language Modelscited by 200
  5. 2022TALM: Tool Augmented Language Modelscited by 197
  6. 2024The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Surveycited by 168
  7. 2023Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Imagescited by 134
  8. 2024HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysiscited by 107
  9. 2023Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videoscited by 86
  10. 2024GTA: A Benchmark for General Tool Agentscited by 77
  11. 2024A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learningcited by 74
  12. 2025DeepAgent: A General Reasoning Agent with Scalable Toolsetscited by 71
  13. 2024MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learningcited by 69
  14. 2023PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of Pathologycited by 57
  15. 2024Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usagecited by 51
  16. 2025LLM Agents Making Agent Toolscited by 48
  17. 2021Structural Persistence in Language Models: Priming as a Window into Abstract Language Representationscited by 44
  18. 2024PathMMU: A Massive Multimodal Expert-Level Benchmark for Understanding and Reasoning in Pathologycited by 43
  19. 2023WsiCaption: Multiple Instance Generation of Pathology Reports for Gigapixel Whole-Slide Imagescited by 31
  20. 2024PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaborationcited by 28
  21. 2024STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomicscited by 23
  22. 2025Disentangling Feature Structure: A Mathematically Provable Two-Stage Training Dynamics in Transformerscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Learning CLI Agents with Structured Action Credit under Selective Observationcited by 1
  2. 2026Democratizing and accelerating AI-driven pathology research through agentic intelligencecited by 1
  3. 2026Evaluating Agentic Bioinformatics through Function, Evidence, and Validationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic
  2. 2026SAGE: Agentic Framework for Interpretable and Clinically Translatable Computational Pathology Biomarker Discovery
  3. 2025PathAgent: Toward Interpretable Analysis of Whole-slide Pathology Images via Large Language Model-based Agentic Reasoning
  4. 2026A Multimodal Agentic Pathology Co-pilot via Evidence Grounded Reasoning
  5. 2026CellDX AI Autopilot: Agent-Guided Training and Deployment of Pathology Classifiers
  6. 2025NOVA: An Agentic Framework for Automated Histopathology Analysis and Discovery
  7. 2026PathoSage: Towards Multi-Source Evidence Adjudication in Pathology via Experience-Aware Agentic Workflow
  8. 2025PathFound: An Agentic Multimodal Model Activating Evidence-seeking Pathological Diagnosis