PolarMem: A Training-Free Polarized Latent Graph Memory for Verifiable Vision-Language Models
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Memory is not merely a storage mechanism for intelligent systems, but a structure for organizing evidence and constraining belief. This is especially important for multimodal reasoning, where retrieved evidence must be both query-relevant and visually consistent. However, current memory systems for vision-language models (VLMs) remain largely positive-associative: they retrieve what is similar or previously observed, but lack an explicit way to remember what has been verified as absent or logically excluded. To this end, we propose PolarMem, a training-free polarized latent graph memory framework for verifiable vision-language reasoning. PolarMem transforms frozen VLM perceptual signals into textitHAS, textitNOTHAS, and textitUncertain memory states through semantic consistency verification and adaptive distributional partitioning, and stores them in a polarized graph with distinct positive and negative memory relations. During inference, a lexicographical logic-aware retrieval protocol enforces logical consistency before semantic similarity, suppressing conflicting memories before they enter the model context. Across eight frozen VLM backbones and six multimodal benchmarks, PolarMem consistently improves retrieval-intensive tasks and reduces retrieval-level contradictions. These results highlight negative memory as a key mechanism for building more reliable multimodal memory systems. Our code is available at https://github.com/czs-ict/PolarMem.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 33 references that are in the map. First posted on arXiv 31 Jan 2026.
Builds on
13 of 33 references.
Closest ancestors · 1
Same direction, newest first
Other references · 12
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
- 2024LightRAG: Simple and Fast Retrieval-Augmented Generationcited by 310
- 2025Memory in the Age of AI Agentscited by 285
- 2025MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulationcited by 183
- 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 88
- 2024MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Modelscited by 41
- 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
- 2025Visual-RAG: Benchmarking Text-to-Image Retrieval Augmented Generation for Visual Knowledge Intensive Queriescited by 16
- 2025Agentic Learner with Grow-and-Refine Multimodal Semantic Memorycited by 11
- 2026FlashMem: Distilling Intrinsic Latent Memory via Computation Reusecited by 7
- 2025Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language Modelscited by 5
- 2025V-ITI: Mitigating Hallucinations in Multimodal Large Language Models via Visual Inference-Time Interventioncited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Dual-Latent Memory Routing for Vision-Language Reasoning
- 2026MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models
- 2026Self-Consistent Latent Reasoning: Long Latent Sequence Reasoning for Vision-Language Model
- 2025Towards General Continuous Memory for Vision-Language Models
- 2026Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning
- 2026Inference-Time Structural Reasoning for Compositional Vision-Language Understanding
- 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
- 2026Grid2Matrix: Revealing Digital Agnosia in Vision-Language Models