MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolution
Not on the map. Why?
Abstract (arXiv)
High-precision medical diagnosis relies not only on static imaging features but also on the implicit diagnostic memory experts instantly invoke during image interpretation. We pinpoint a fundamental cognitive misalignment in medical VLMs caused by discrete tokenization, leading to quantization loss, long-range information dissipation, and missing case-adaptive expertise. To bridge this gap, we propose ours, a framework for latent diagnostic memory evolution that simulates the experiential invocation of clinicians by dynamically synthesizing implicit diagnostic memories within the model's hidden stream. Specifically, it begins with a Meta Query for Prior Memorization mechanism, where learnable probes retrieve structured priors from an anatomical prior encoder to generate condensed implicit memories. To ensure clinical fidelity, we introduce Causal Counterfactual Refinement (CCR), which leverages reinforcement learning and counterfactual rewards derived from region-level feature masking to quantify the causal contribution of each memory, thereby pruning redundancies and aligning latent representations with diagnostic logic. This evolutionary process culminates in Intrinsic Memory Transition (IMT), a privileged-autonomous dual-branch paradigm that internalizes teacher-branch diagnostic patterns into the student-branch via full-vocabulary divergence alignment. Comprehensive empirical evaluations across multiple datasets demonstrate that ours, by transferring external expertise into endogenous parameters, significantly outperforms existing state-of-the-art methods, particularly chain-of-thought paradigms, in diagnostic accuracy. The code is available at https://github.com/zhcz328/MedSynapse-V.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 146 of its 220 references that are in the map. First posted on arXiv 29 Apr 2026.
Builds on
146 of 220 references.
References · 146
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
38 more
- 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
- 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
- 2025MedGemma Technical Reportcited by 398
- 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
- 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
- 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 225
- 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
- 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 212
- 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 182
- 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learningcited by 172
- 2023SAM-Med2Dcited by 123
- 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicinecited by 122
- 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMscited by 110
- 2024MMedAgent: Learning to Use Medical Tools with Multi-modal Agentcited by 108
- 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphscited by 105
- 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 99
- 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
- 2025Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understandingcited by 96
- 2024Medical Graph RAG: Towards Safe Medical Large Language Model via Graph Retrieval-Augmented Generationcited by 89
- 2025MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilotcited by 83
- 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chainscited by 81
- 2024Parameter Competition Balancing for Model Mergingcited by 71
- 2023Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Modelscited by 58
- 2024Detecting and Evaluating Medical Hallucinations in Large Vision Language Modelscited by 57
- 2024VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledgecited by 50
- 2025Reinforced Latent Reasoning for LLM-based Recommendationcited by 46
- 2025Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Spacecited by 40
- 2025GMAI-VL-R1: Harnessing Reinforcement Learning for Multimodal Medical Reasoningcited by 39
- 2025SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoningcited by 38
- 2025Evaluating and Steering Modality Preferences in Multimodal Large Language Modelcited by 33
- 2025MedEyes: Learning Dynamic Visual Focus for Medical Progressive Diagnosiscited by 32
- 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 31
- 2025Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMscited by 30
- 2025Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Modelscited by 29
- 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Drivingcited by 28
- 2026MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methodscited by 27
- 2024CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generationcited by 25
50 most cited of 146 All 146 (JSON)
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026ChainCaps: Composition-Safe Tool-Using Agents via Monotonic Capability Attenuationcited by 11
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rolloutcited by 6
- 2026CoLT: Teaching Multi-Modal Models to Think with Chain of Latent Thoughtscited by 3
- 2026Deformba: Vision State Space Model with Adaptive State Fusioncited by 2
- 2026Dynamic Model Merging Made Slimcited by 1
- 2026Skill Weaving: Efficient LLM Improvement via Modular Skillpackscited by 1
- 2026REFLEX: Reflective Evolution from LLM Experiencecited by 1
- 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomycited by 1
- 2026MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agentscited by 1
- 2026All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMscited by 1
- 2026TopoAgent: An Agentic Framework for Automated Topology Learning in Medical Imagingcited by 0
- 2026AtomRec: Evolving Atomic Memory for Agentic Recommendationcited by 0
- 2026All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs5 Sep 2026
- 2026AtomRec: Evolving Atomic Memory for Agentic Recommendation4 Sep 2026
- 2026MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents27 Jul 2026
- 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy13 Jul 2026
- 2026CoLT: Teaching Multi-Modal Models to Think with Chain of Latent Thoughts30 Jun 2026
- 2026TopoAgent: An Agentic Framework for Automated Topology Learning in Medical Imaging29 Jun 2026
- 2026REFLEX: Reflective Evolution from LLM Experience15 Jun 2026
- 2026ChainCaps: Composition-Safe Tool-Using Agents via Monotonic Capability Attenuation26 May 2026
- 2026Skill Weaving: Efficient LLM Improvement via Modular Skillpacks21 May 2026
- 2026Deformba: Vision State Space Model with Adaptive State Fusion20 May 2026
- 2026Dynamic Model Merging Made Slim17 May 2026
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout6 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026MedExpMem: Adapting Experience Memory for Differential Diagnosis
- 2026MSM-Mem: A Universal Medical Structured Multimodal Memory Framework for Medical AI Agents
- 2026Evolving Medical Imaging Agents via Experience-driven Self-skill Discovery
- 2026Making medical vision-language models think causally across modalities with retrieval-augmented cross-modal reasoning
- 2026SAGEAgent: A Self-Evolving Agent for Cost-Aware Modality Acquisition in Multimodal Survival Prediction
- 2026Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improve
- 2026MIRA: Medical Image Reflection for Agentic Diagnosis
- 2025Embeddings to Diagnosis: Latent Fragility under Agentic Perturbations in Clinical LLMs