# Context-Aware Multi-Turn Visual-Textual Reasoning in LVLMs via Dynamic Memory and Adaptive Visual Guidance

arXiv: https://arxiv.org/abs/2509.05669 · HTML: https://mlsanity.com/paper/2509.05669/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 6 Sep 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 25 of 25)

- [Visual In-Context Learning for Large Vision-Language Models](https://mlsanity.com/paper/2402.11574/) (2024)
- [Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey](https://mlsanity.com/paper/2412.20367/) (2024)
- [GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems](https://mlsanity.com/paper/2010.03994/) (2020)
- [Improving Medical Large Vision-Language Models with Abnormal-Aware Feedback](https://mlsanity.com/paper/2501.01377/) (2025)
- [SCORE: Story Coherence and Retrieval Enhancement for AI Narratives](https://mlsanity.com/paper/2503.23512/) (2025)
- [SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understanding](https://mlsanity.com/paper/2503.18943/) (2025)
- [ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies](https://mlsanity.com/paper/2506.12830/) (2025)
- [Question Relevance in VQA: Identifying Non-Visual And False-Premise Questions](https://mlsanity.com/paper/1606.06622/) (2016)
- [MMCR: Advancing Visual Language Model in Multimodal Multi-Turn Contextual Reasoning](https://mlsanity.com/paper/2503.18533/) (2025)
- [Demonstrating and Reducing Shortcuts in Vision-Language Representation Learning](https://mlsanity.com/paper/2402.17510/) (2024)
- [Learning Reasoning Paths over Semantic Graphs for Video-grounded Dialogues](https://mlsanity.com/paper/2103.00820/) (2021)
- [Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architectures](https://mlsanity.com/paper/2508.10824/) (2025)
- [The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training](https://mlsanity.com/paper/2205.12502/) (2022)
- [Beyond the Granularity: Multi-Perspective Dialogue Collaborative Selection for Dialogue State Tracking](https://mlsanity.com/paper/2205.10059/) (2022)
- [Conformalized Deep Splines for Optimal and Efficient Prediction Sets](https://mlsanity.com/paper/2311.00774/) (2023)
- [Memory and attention in deep learning](https://mlsanity.com/paper/2107.01390/) (2021)
- [DSBERT:Unsupervised Dialogue Structure learning with BERT](https://mlsanity.com/paper/2111.04933/) (2021)
- [Towards a Unified Model for Generating Answers and Explanations in Visual Question Answering](https://mlsanity.com/paper/2301.10799/) (2023)
- [DialogueReason: Rule-Based RL Sparks Dialogue Reasoning in LLMs](https://mlsanity.com/paper/2505.07049/) (2025)
- [Improved Attention Models for Memory Augmented Neural Network Adaptive Controllers](https://mlsanity.com/paper/1910.01189/) (2019)
- [Modelling of AC/DC Interactions of Converter-Interfaced Resources for Harmonic Power-Flow Studies in Microgrids](https://mlsanity.com/paper/2206.07332/) (2022)
- [Evaluating Scenario-based Decision-making for Interactive Autonomous Driving Using Rational Criteria: A Survey](https://mlsanity.com/paper/2501.01886/) (2025)
- [Attention is All You Need Until You Need Retention](https://mlsanity.com/paper/2501.09166/) (2025)
- [A Neural Network Model of Spatial and Feature-Based Attention](https://mlsanity.com/paper/2506.05487/) (2025)
- [Sequence-to-Sequence Models with Attention Mechanistically Map to the Architecture of Human Memory Search](https://mlsanity.com/paper/2506.17424/) (2025)

## Built on by (most cited first, 1 of 1)

- [Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs](https://mlsanity.com/paper/2605.00814/) (2026)

## Abstract (arXiv)

Current Large Language Models (LLMs) and Vision-Language Large Models (LVLMs) excel in single-turn tasks but face significant challenges in multi-turn interactions requiring deep contextual understanding and complex visual reasoning, often leading to fragmented reasoning, context loss, and hallucinations. To address these limitations, we propose Context-Aware Multi-Turn Visual Reasoning (CAMVR), a novel framework designed to empower LVLMs with robust and coherent multi-turn visual-textual inference capabilities. CAMVR introduces two key innovations: a Visual-Textual Context Memory Unit (VCMU), a dynamic read-write memory network that stores and manages critical visual features, textual semantic representations, and their cross-modal correspondences from each interaction turn; and an Adaptive Visual Focus Guidance (AVFG) mechanism, which leverages the VCMU's context to dynamically adjust the visual encoder's attention to contextually relevant image regions. Our multi-level reasoning integration strategy ensures that response generation is deeply coherent with both current inputs and accumulated historical context. Extensive experiments on challenging datasets, including VisDial, an adapted A-OKVQA, and our novel Multi-Turn Instruction Following (MTIF) dataset, demonstrate that CAMVR consistently achieves state-of-the-art performance.