Scene-Aware Vectorized Memory Multi-Agent Framework with Cross-Modal Differentiated Quantization VLMs for Visually Impaired Assistance
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Visually impaired individuals face significant challenges in environmental perception. Traditional assistive technologies often lack adaptive intelligence, focusing on individual components rather than integrated systems. While Vision-Language Models (VLMs) offer a promising path to richer, integrated understanding, their deployment is severely limited by substantial computational requirements, demanding dozens of gigabytes of memory. To address these gaps in computational efficiency and integrated design, this study proposes a dual technological innovation framework: a cross-modal differentiated quantization framework for VLMs and a scene-aware vectorized memory multi-agent system. The quantization framework implements differentiated strategies, reducing memory from 38GB to 11.3GB. The multi-agent system uses vectorized memory and perception-memory-reasoning workflows to provide environmental information beyond the current view, achieving 2.83-3.52s latency to initial speech output. Experiments show the quantized 19B-parameter model only experiences a 2.05% performance drop on MMBench and maintains 63.7 accuracy on OCR-VQA (original: 64.9), outperforming smaller models with equivalent memory. This research advances computational efficiency and assistive technology, offering comprehensive assistance in scene perception, text recognition, and navigation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 60 references that are in the map. First posted on arXiv 25 Aug 2025.
Builds on
35 of 60 references.
References · 35
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2023Qwen Technical Reportcited by 3,344
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
- 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
- 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
- 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
23 more
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
- 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
- 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 598
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
- 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
- 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
- 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
- 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 217
- 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Aheadcited by 155
- 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 97
- 2024BRAVE: Broadening the visual encoding of vision-language modelscited by 69
- 2024Q-VLM: Post-training Quantization for Large Vision-Language Modelscited by 47
- 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Modelscited by 45
- 2024From LLM to Conversational Agent: A Memory Enhanced Architecture with Fine-Tuning of Large Language Modelscited by 43
- 2024MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answeringcited by 43
- 2024Exploration of LLM Multi-Agent Application Implementation Based on LangGraph+CrewAIcited by 35
- 2025LexRAG: Benchmarking Retrieval-Augmented Generation in Multi-Turn Legal Consultation Conversationcited by 19
- 2024Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documentscited by 17
- 2024Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmascited by 12
- 2025SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Modelscited by 8
- 2024Evaluation and Comparison of Visual Language Models for Transportation Engineering Problemscited by 4
- 2025QUAD: Quantization and Parameter-Efficient Tuning of LLM with Activation Decompositioncited by 4
- 2025VocalEyes: Enhancing Environmental Perception for the Visually Impaired through Vision-Language Models and Distance-Aware Object Detectioncited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-use
- 2026Collaborative Memory for Multi-Agent VLM Systems
- 2026VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models
- 2024InsightSee: Advancing Multi-agent Vision-Language Models for Enhanced Visual Understanding
- 2026LQA: A Lightweight Quantized-Adaptive Framework for Vision-Language Models on the Edge
- 2026MAIL++: Multi-Modal Bi-directional Agent Layer for Vision-Language Models
- 2024Visual Language Model based Cross-modal Semantic Communication Systems
- 2025VLURes: Benchmarking VLM Visual and Linguistic Understanding in Low-Resource Languages