mlsanityData to 14 Oct 2026

Cache-of-Thought: Master-Apprentice Framework for Cost-Effective Vision Language Model Reasoning

arXiv 2502.20587 ↗First posted 27 Feb 2025cs.LGcited by 3

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Vision Language Models (VLMs) have achieved remarkable success in a wide range of vision applications of increasing complexity and scales, yet choosing the right VLM model size involves a trade-off between response quality and cost. While smaller VLMs are cheaper to run, they typically produce responses only marginally better than random guessing on benchmarks such as MMMU. In this paper, we propose Cache of Thought (CoT), a master apprentice framework for collaborative inference between large and small VLMs. CoT manages high quality query results from large VLMs (master) in a cache, which are then selected via a novel multi modal retrieval and in-context learning to aid the performance of small VLMs (apprentice). We extensively evaluate CoT on various widely recognized and challenging general reasoning benchmarks, and show that CoT increases overall reasoning performance by up to 7.7% under the same budget, and specifically boosts the performance of apprentice VLMs by up to 36.6%. Our code is available at https://github.com/UIUC-MONET/Cache-of-Thoughts

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 46 references that are in the map. First posted on arXiv 27 Feb 2025.

Builds on

14 of 46 references.

References · 14

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  6. 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Modelcited by 174
  7. 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devicescited by 122
  8. 2024Manipulate-Anything: Automating Real-World Robots using Vision-Language Modelscited by 83
  9. 2024Many-Shot In-Context Learning in Multimodal Foundation Modelscited by 67
  10. 2024Yo'LLaVA: Your Personalized Language and Vision Assistantcited by 62
  11. 2024MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Trainingcited by 50
  12. 2024VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learningcited by 36
2 more
  1. 2024Rec-GPT4V: Multimodal Recommendation with Large Vision-Language Modelscited by 18
  2. 2024Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Enginescited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Usecited by 90
  2. 2025Semantic Caching of Contextual Summaries for Efficient Question-Answering with Language Modelscited by 5
  3. 2026PAIR: Memory-Efficient and Fast Inference with Reading--Reasoning Fusioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters
  2. 2023Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models
  3. 2024Improve Vision Language Model Chain-of-thought Reasoning
  4. 2023IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models
  5. 2024UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling
  6. 2026Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
  7. 2024VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration
  8. 2025AirCache: Activating Inter-modal Relevancy KV Cache Compression for Efficient Large Vision-Language Model Inference