mlsanityData to 7 Oct 2026

CoCoT: Contrastive Chain-of-Thought Prompting for Large Multimodal Models with Multiple Image Inputs

arXiv 2401.02582 ↗First posted 5 Jan 2024cs.CVcited by 56

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

When exploring the development of Artificial General Intelligence (AGI), a critical task for these models involves interpreting and processing information from multiple image inputs. However, Large Multimodal Models (LMMs) encounter two issues in such scenarios: (1) a lack of fine-grained perception, and (2) a tendency to blend information across multiple images. We first extensively investigate the capability of LMMs to perceive fine-grained visual details when dealing with multiple input images. The research focuses on two aspects: first, image-to-image matching (to evaluate whether LMMs can effectively reason and pair relevant images), and second, multi-image-to-text matching (to assess whether LMMs can accurately capture and summarize detailed image information). We conduct evaluations on a range of both open-source and closed-source large models, including GPT-4V, Gemini, OpenFlamingo, and MMICL. To enhance model performance, we further develop a Contrastive Chain-of-Thought (CoCoT) prompting approach based on multi-input multimodal models. This method requires LMMs to compare the similarities and differences among multiple image inputs, and then guide the models to answer detailed questions about multi-image inputs based on the identified similarities and differences. Our experimental results showcase CoCoT's proficiency in enhancing the multi-image comprehension capabilities of large multimodal models.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 23 references that are in the map. First posted on arXiv 5 Jan 2024.

Builds on

17 of 23 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Compositional Chain-of-Thought Prompting for Large Multimodal Models27 Nov
  2. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models25 Oct
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  4. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality7 Apr

Other references · 13

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  7. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 225
  8. 2023MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learningcited by 171
  9. 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
  10. 2022PromptCap: Prompt-Guided Task-Aware Image Captioningcited by 104
  11. 2023Factify 2: A Multimodal Fake News and Satire News Datasetcited by 23
  12. 2022Unbiased Multi-Modality Guidance for Image Inpaintingcited by 10
1 more
  1. 2023LLMVA-GEBC: Large Language Model with Video Adapter for Generic Event Boundary Captioningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 25; 2025: 26; 2026: 5.

56 papers.

Sort:

By area:

  1. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  2. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  3. 2024Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Modelscited by 64
  4. 2025A Survey on (M)LLM-Based GUI Agentscited by 61
  5. 2024MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMscited by 60
  6. 2025Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMscited by 47
  7. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Rewardcited by 45
  8. 2024TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understandingcited by 37
  9. 2024Cantor: Inspiring Multimodal Chain-of-Thought of MLLMcited by 34
  10. 2025Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thoughtcited by 34
  11. 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrievalcited by 31
  12. 2024What is the Visual Cognition Gap between Humans and Multimodal LLMs?cited by 30
38 more
  1. 2024Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucinationcited by 29
  2. 2024CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Modelscited by 29
  3. 2025WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learningcited by 21
  4. 2024An Empirical Analysis on Large Language Models in Debate Evaluationcited by 20
  5. 2025Can Multimodal Large Language Models Understand Spatial Relations?cited by 20
  6. 2024Empowering LLMs with Pseudo-Untrimmed Videos for Audio-Visual Temporal Understandingcited by 18
  7. 2024Img-Diff: Contrastive Data Synthesis for Multimodal Large Language Modelscited by 17
  8. 2024FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correctioncited by 16
  9. 2024EAGLE: Egocentric AGgregated Language-video Enginecited by 16
  10. 2024IW-Bench: Evaluating Large Multimodal Models for Converting Image-to-Webcited by 16
  11. 2025Benchmarking Multimodal CoT Reward Model Stepwise by Visual Programcited by 10
  12. 2025Visual Reasoning Tracer: Object-Level Grounded Reasoning Benchmarkcited by 10
  13. 2025Look Less, Reason More: Rollout-Guided Adaptive Pixel-Space Reasoningcited by 9
  14. 2024Natural Language Inference Improves Compositionality in Vision-Language Modelscited by 8
  15. 2025VISTA: Mitigating Semantic Inertia in Video-LLMs via Training-Free Dynamic Chain-of-Thought Routingcited by 7
  16. 2024X-Reflect: Cross-Reflection Prompting for Multimodal Recommendationcited by 6
  17. 2024Chain-of-Thought in Large Language Models: Decoding, Projection, and Activationcited by 6
  18. 2024MageBench: Bridging Large Multimodal Models to Agentscited by 6
  19. 2024MineAgent: Towards Remote-Sensing Mineral Exploration with Multimodal Large Language Modelscited by 6
  20. 2025Towards Comprehensive Scene Understanding: Integrating First and Third-Person Views for LVLMscited by 6
  21. 2025Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Modelscited by 6
  22. 2024Harnessing GPT-4V(ision) for Insurance: A Preliminary Explorationcited by 5
  23. 2024Smart Vision-Language Reasonerscited by 5
  24. 2024Autonomous Imagination: Closed-Loop Decomposition of Visual-to-Textual Conversion in Visual Reasoning for Multimodal Large Language Modelscited by 5
  25. 2025A Cognitive Paradigm Approach to Probe the Perception-Reasoning Interface in VLMscited by 5
  26. 2024BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Modelscited by 4
  27. 2024Chain-of-Thought Prompting for Demographic Inference with Large Multimodal Modelscited by 4
  28. 2025Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understandingcited by 3
  29. 2026TIR-Flow: Active Video Search and Reasoning with Frozen VLMscited by 3
  30. 2025SCRAMBLe : Enhancing Multimodal LLM Compositionality with Synthetic Preference Datacited by 2
  31. 2025PathCoT: Chain-of-Thought Prompting for Zero-shot Pathology Visual Reasoningcited by 2
  32. 2025From Answers to Rationales: Self-Aligning Multimodal Reasoning with Answer-Oriented Chain-of-Thoughtcited by 2
  33. 2025PDB-Eval: An Evaluation of Large Multimodal Models for Description and Explanation of Personalized Driving Behaviorcited by 2
  34. 2026SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Modelscited by 2
  35. 2025Be a Multitude to Itself: A Prompt Evolution Framework for Red Teamingcited by 1
  36. 2025Visual CoT Makes VLMs Smarter but More Fragilecited by 1
  37. 2025QG-CoC: Question-Guided Chain-of-Captions for Large Multimodal Modelscited by 1
  38. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)
  2. 2023Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Framework
  3. 2026CGC: Compositional Grounded Contrast for Fine-Grained Multi-Image Understanding
  4. 2024Interleaved-Modal Chain-of-Thought
  5. 2023Multimodal Analysis Of Google Bard And GPT-Vision: Experiments In Visual Reasoning
  6. 2023Large Multimodal Models: Notes on CVPR 2023 Tutorial
  7. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers
  8. 2026TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding