# The Role of Chain-of-Thought in Complex Vision-Language Reasoning Task

arXiv: https://arxiv.org/abs/2311.09193 · HTML: https://mlsanity.com/paper/2311.09193/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 8 of 8)

- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning](https://mlsanity.com/paper/2310.09478/) (2023)
- [Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://mlsanity.com/paper/2310.11441/) (2023)
- [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023)
- [MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning](https://mlsanity.com/paper/2309.07915/) (2023)
- [What You See is What You Read? Improving Text-Image Alignment Evaluation](https://mlsanity.com/paper/2305.10400/) (2023)

## Built on by (most cited first, 30 of 32)

- [Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2503.12605/) (2025)
- [PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding](https://mlsanity.com/paper/2501.16411/) (2025)
- [M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought](https://mlsanity.com/paper/2405.16473/) (2024)
- [Large Language Models Meet NLP: A Survey](https://mlsanity.com/paper/2405.12819/) (2024)
- [Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Model](https://mlsanity.com/paper/2501.07246/) (2025)
- [A Survey on (M)LLM-Based GUI Agents](https://mlsanity.com/paper/2504.13865/) (2025)
- [Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models](https://mlsanity.com/paper/2402.12058/) (2024)
- [DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment](https://mlsanity.com/paper/2507.02768/) (2025)
- [SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models](https://mlsanity.com/paper/2402.04178/) (2024)
- [VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Information](https://mlsanity.com/paper/2412.00947/) (2024)
- [CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models](https://mlsanity.com/paper/2412.12932/) (2024)
- [Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models](https://mlsanity.com/paper/2408.00523/) (2024)
- [Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models](https://mlsanity.com/paper/2407.11422/) (2024)
- [ColorSwap: A Color and Word Order Dataset for Multimodal Evaluation](https://mlsanity.com/paper/2402.04492/) (2024)
- [Natural Language Inference Improves Compositionality in Vision-Language Models](https://mlsanity.com/paper/2410.22315/) (2024)
- [Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Models](https://mlsanity.com/paper/2510.07632/) (2025)
- [SDRT: Enhance Vision-Language Models by Self-Distillation with Diverse Reasoning Traces](https://mlsanity.com/paper/2503.01754/) (2025)
- [GODBench: A Benchmark for Multimodal Large Language Models in Video Comment Art](https://mlsanity.com/paper/2505.11436/) (2025)
- [BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models](https://mlsanity.com/paper/2402.13577/) (2024)
- [MathOPEval: A Fine-grained Evaluation Benchmark for Visual Operations of MLLMs in Mathematical Reasoning](https://mlsanity.com/paper/2507.18140/) (2025)
- [Image First or Text First? Optimising the Sequencing of Modalities in Large Language Model Prompting and Reasoning Tasks](https://mlsanity.com/paper/2410.03062/) (2024)
- [Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Events](https://mlsanity.com/paper/2603.06213/) (2026)
- [Exploring Multi-Grained Concept Annotations for Multimodal Large Language Models](https://mlsanity.com/paper/2412.05939/) (2024)
- [From Answers to Rationales: Self-Aligning Multimodal Reasoning with Answer-Oriented Chain-of-Thought](https://mlsanity.com/paper/2507.02984/) (2025)
- [Visual Language Models as Zero-Shot Deepfake Detectors](https://mlsanity.com/paper/2507.22469/) (2025)
- [MIND: Multi-rationale INtegrated Discriminative Reasoning Framework for Multi-modal Large Models](https://mlsanity.com/paper/2512.05530/) (2025)
- [Improving Reasoning in Vision-Language Models via Perception Verified Self-Training](https://mlsanity.com/paper/2606.22158/) (2026)
- [Chain-of-Description: What I can understand, I can put into words](https://mlsanity.com/paper/2502.16137/) (2025)
- [Large Language Models Facilitate Vision Reflection in Image Classification](https://mlsanity.com/paper/2508.06525/) (2025)
- [Conversational Orientation Reasoning: Egocentric-to-Allocentric Navigation with Multimodal Chain-of-Thought](https://mlsanity.com/paper/2509.18200/) (2025)

## Abstract (arXiv)

The study explores the effectiveness of the Chain-of-Thought approach, known for its proficiency in language tasks by breaking them down into sub-tasks and intermediate steps, in improving vision-language tasks that demand sophisticated perception and reasoning. We present the "Description then Decision" strategy, which is inspired by how humans process signals. This strategy significantly improves probing task performance by 50%, establishing the groundwork for future research on reasoning paradigms in complex vision-language tasks.