# EndoCogniAgent: Closed-Loop Agentic Reasoning with Self-Consistency Validation for Endoscopic Diagnosis arXiv: https://arxiv.org/abs/2508.07292 ยท HTML: https://mlsanity.com/paper/2508.07292/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. - Builds on 29 of its 55 references that are in the map. - First posted on arXiv 10 Aug 2025. ## Placed in - LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/) - LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/) ## Builds on (most cited first, 29 of 29) - [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022) - [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022) - [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022) - [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024) - [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025) - [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022) - [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023) - [Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs](https://mlsanity.com/paper/2401.06209/) (2024) - [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024) - [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025) - [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019) - [Med-Flamingo: a Multimodal Medical Few-shot Learner](https://mlsanity.com/paper/2307.15189/) (2023) - [AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments](https://mlsanity.com/paper/2405.07960/) (2024) - [HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale](https://mlsanity.com/paper/2406.19280/) (2024) - [MMedAgent: Learning to Use Medical Tools with Multi-modal Agent](https://mlsanity.com/paper/2407.02483/) (2024) - [Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train](https://mlsanity.com/paper/2306.16741/) (2023) - [MedRAX: Medical Reasoning Agent for Chest X-ray](https://mlsanity.com/paper/2502.02673/) (2025) - [NeoUNet: Towards accurate colon polyp segmentation and neoplasm detection](https://mlsanity.com/paper/2107.05023/) (2021) - [Kvasir-VQA: A Text-Image Pair GI Tract Dataset](https://mlsanity.com/paper/2409.01437/) (2024) - [GastroVision: A Multi-class Endoscopy Image Dataset for Computer Aided Gastrointestinal Disease Detection](https://mlsanity.com/paper/2307.08140/) (2023) - [Frontiers in Intelligent Colonoscopy](https://mlsanity.com/paper/2410.17241/) (2024) - [EndoMamba: An Efficient Foundation Model for Endoscopic Videos via Hierarchical Pre-training](https://mlsanity.com/paper/2502.19090/) (2025) - [FFCNet: Fourier Transform-Based Frequency Learning and Complex Convolutional Network for Colon Disease Classification](https://mlsanity.com/paper/2207.01287/) (2022) - [EndoDINO: A Foundation Model for GI Endoscopy](https://mlsanity.com/paper/2501.05488/) (2025) - [Polyp-Gen: Realistic and Diverse Polyp Image Generation for Endoscopic Dataset Expansion](https://mlsanity.com/paper/2501.16679/) (2025) - [MedOrch: Medical Diagnosis with Tool-Augmented Reasoning Agents for Flexible Extensibility](https://mlsanity.com/paper/2506.00235/) (2025) - [Colorectal Polyp Classification from White-light Colonoscopy Images via Domain Alignment](https://mlsanity.com/paper/2108.02476/) (2021) - [MedMax: Mixed-Modal Instruction Tuning for Training Biomedical Assistants](https://mlsanity.com/paper/2412.12661/) (2024) - [Universal Medical Image Representation Learning with Compositional Decoders](https://mlsanity.com/paper/2409.19890/) (2024) ## Built on by (most cited first, 3 of 3) - [SurvAgent: Hierarchical CoT-Enhanced Case Banking and Dichotomy-Based Multi-Agent System for Multimodal Survival Prediction](https://mlsanity.com/paper/2511.16635/) (2025) - [Picking the Right Specialist: Attentive Neural Process-based Selection of Task-Specialized Models as Tools for Agentic Healthcare Systems](https://mlsanity.com/paper/2602.14901/) (2026) - [Measuring and Improving Complex-Atomic Answer Consistency in Endoscopic VQA](https://mlsanity.com/paper/2607.17834/) (2026) ## Abstract (arXiv) Endoscopic diagnosis is an iterative process in which clinicians acquire, compare, and verify local visual evidence before reaching a conclusion. Current AI systems do not adequately support this process because fine-grained evidence acquisition and multi-step reasoning remain weakly coupled, complicating reconciliation of image-derived findings with their textual interpretations. This gives rise to two failure modes, hallucinated evidence and uncorrected error accumulation, that undermine diagnostic reliability. We propose EndoCogniAgent, a closed-loop agentic framework that formulates endoscopic diagnosis as a controlled state update process for integrating complementary visual and textual evidence. At each reasoning round, a central planner selects an evidence acquisition action, specialized expert tools extract spatial and semantic observations as structured textual evidence, and a self-consistency validation mechanism examines this evidence along two dimensions, knowledge consistency against the input image and temporal consistency with prior validated findings, before updating the diagnostic state. Validated observations are admitted into the evolving state to condition subsequent planning, while insufficiently supported or conflicting findings are retained with corrective feedback that redirects the planner toward additional verification. We further introduce EndoAgentBench, a workflow-oriented benchmark comprising 6,132 question-answer pairs from 11 endoscopic datasets, to evaluate diagnostic agents across a comprehensive diagnostic chain, from fine-grained visual perception to high-level diagnostic reasoning. EndoCogniAgent achieves 85.23% overall accuracy on perception tasks and 71.13% clinical acceptance rate on reasoning tasks. Blinded clinician evaluation further shows consistent improvements in diagnostic response quality over the evaluated baselines.