mlsanityData to 7 Oct 2026

Chain of Thought Prompt Tuning in Vision Language Models

arXiv 2304.07919 ↗First posted 16 Apr 2023cs.CV cs.AIcited by 34

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Language-Image Pre-training has demonstrated promising results on zero-shot and few-shot downstream tasks by prompting visual models with natural language prompts. However, most recent studies only use a single prompt for tuning, neglecting the inherent step-to-step cognitive reasoning process that humans conduct in complex task settings, for example, when processing images from unfamiliar domains. Chain of Thought is a simple and effective approximation to human reasoning process and has been proven useful for natural language processing (NLP) tasks. Based on this cognitive intuition, we believe that conducting effective reasoning is also an important problem in visual tasks, and a chain of thought could be a solution to this problem. In this work, we propose a novel chain of thought prompt tuning for vision-language modeling. Extensive experiments show that our method not only generalizes better in image classification tasks, has greater transferability beyond a single dataset, and has stronger domain generalization performance, but also performs much better in imagetext retrieval and visual question answering, which require more reasoning capabilities. We are the first to successfully adapt chain-of-thought prompting that combines visual and textual embeddings. We will release our codes

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 8 of its 43 references that are in the map. First posted on arXiv 16 Apr 2023.

Builds on

8 of 43 references.

References · 8

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  3. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  4. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  5. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  6. 2022Prompt-aligned Gradient for Prompt Tuningcited by 339
  7. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  8. 2022Language models show human-like content effects on reasoning taskscited by 171

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 13; 2025: 8; 2026: 5.

34 papers.

Sort:

By area:

  1. 2023A Survey on Multimodal Large Language Modelscited by 898
  2. 2023A Comprehensive Overview of Large Language Modelscited by 639
  3. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  4. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  5. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  6. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 92
  7. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  8. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
  9. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  10. 2023BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Modelscited by 41
  11. 2024Autoregressive Models in Vision: A Surveycited by 39
  12. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
22 more
  1. 2025Argus: Vision-Centric Reasoning with Grounded Chain-of-Thoughtcited by 26
  2. 2024CoDA: Instructive Chain-of-Domain Adaptation with Severity-Aware Visual Prompt Tuningcited by 13
  3. 2024Moderating Illicit Online Image Promotion for Unsafe User-Generated Content Games Using Large Vision-Language Modelscited by 13
  4. 2024Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Modelscited by 12
  5. 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
  6. 2024VEGA: Learning Interleaved Image-Text Comprehension in Vision-Language Large Modelscited by 8
  7. 2024Benchmarking XAI Explanations with Human-Aligned Evaluationscited by 8
  8. 2023Masked Diffusion with Task-awareness for Procedure Planning in Instructional Videoscited by 7
  9. 2024Soft-Prompting with Graph-of-Thought for Multi-modal Representation Learningcited by 7
  10. 2025Harnessing Chain-of-Thought Reasoning in Multimodal Large Language Models for Face Anti-Spoofingcited by 5
  11. 2024A Training-Free Framework for Video License Plate Tracking and Recognition with Only One-Shotcited by 2
  12. 2025XiHeFusion: Harnessing Large Language Models for Science Communication in Nuclear Fusioncited by 2
  13. 2026Multi-scale hypergraph meets LLMs: Aligning large language models for time series analysiscited by 2
  14. 2023Gradient constrained sharpness-aware prompt learning for vision-language modelscited by 1
  15. 2024CPA-Enhancer: Chain-of-Thought Prompted Adaptive Enhancer for Object Detection under Unknown Degradationscited by 1
  16. 2023Trustworthy Large Models in Vision: A Surveycited by 0
  17. 2025Salience Adjustment for Context-Based Emotion Recognitioncited by 0
  18. 2025Q-Doc: Benchmarking Document Image Quality Assessment Capabilities in Multi-modal Large Language Modelscited by 0
  19. 2026Eyes on VLM: Benchmarking Gaze Following and Social Gaze Prediction in Vision Language Modelscited by 0
  20. 2026AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoringcited by 0
  21. 2026Can Vision-Language Models Assess Proxemic Risk from Egocentric Robot Images?cited by 0
  22. 2026Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoning
  2. 2024Interleaved-Modal Chain-of-Thought
  3. 2025Unveiling Chain of Step Reasoning for Vision-Language Models with Fine-grained Rewards
  4. 2025Retrieval-Based Interleaved Visual Chain-of-Thought in Real-World Driving Scenarios
  5. 2023Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models
  6. 2025Rationale-Enhanced Decoding for Multi-modal Chain-of-Thought
  7. 2026Better Eyes, Better Thoughts: Why Vision Chain-of-Thought Fails in Medicine
  8. 2024Chain-of-Thought Augmentation with Logit Contrast for Enhanced Reasoning in Language Models