mlsanityData to 7 Oct 2026

Enhancing Semantics in Multimodal Chain of Thought via Soft Negative Sampling

arXiv 2405.09848 ↗First posted 16 May 2024cs.CL cs.AIcited by 8

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Chain of thought (CoT) has proven useful for problems requiring complex reasoning. Many of these problems are both textual and multimodal. Given the inputs in different modalities, a model generates a rationale and then uses it to answer a question. Because of the hallucination issue, the generated soft negative rationales with high textual quality but illogical semantics do not always help improve answer accuracy. This study proposes a rationale generation method using soft negative sampling (SNSE-CoT) to mitigate hallucinations in multimodal CoT. Five methods were applied to generate soft negative samples that shared highly similar text but had different semantics from the original. Bidirectional margin loss (BML) was applied to introduce them into the traditional contrastive learning framework that involves only positive and negative samples. Extensive experiments on the ScienceQA dataset demonstrated the effectiveness of the proposed method. Code and data are released at https://github.com/zgMin/SNSE-CoT.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 10 of its 79 references that are in the map. First posted on arXiv 16 May 2024.

Builds on

10 of 79 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb

Other references · 9

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  4. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  5. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  6. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  7. 2022Large Language Models Are Reasoning Teacherscited by 318
  8. 2022Teaching Small Language Models to Reasoncited by 279
  9. 2022SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samplescited by 39

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  2. 2026SceneAlign: Aligning Multimodal Reasoning to Scene Graphs in Complex Visual Scenescited by 11
  3. 2025Beyond Spurious Signals: Debiasing Multimodal Large Language Models via Counterfactual Inference and Adaptive Expert Routingcited by 5
  4. 2025Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identificationcited by 3
  5. 2026EVL-MCoT: Enhanced Vision-Language Multi-CoT for Harmful Meme Detectioncited by 1
  6. 2025Rationale-Enhanced Decoding for Multi-modal Chain-of-Thoughtcited by 0
  7. 2026Visual Saliency Steering Distillation for Multimodal Chain-of-Thought Reasoningcited by 0
  8. 2026DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Seeing Through the Chain: Mitigate Hallucination in Multimodal Reasoning Models via CoT Compression and Contrastive Preference Optimization
  2. 2026Understanding and Mitigating Hallucinations in Multimodal Chain-of-Thought Models
  3. 2025Grounded Chain-of-Thought for Multimodal Large Language Models
  4. 2025Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking
  5. 2025Cross Domain Evaluation of Multimodal Chain-of-Thought Reasoning of different datasets into the Amazon CoT Framework
  6. 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do
  7. 2025Improving the Reliability of LLMs: Combining CoT, RAG, Self-Consistency, and Self-Verification
  8. 2026Reasoning Matters: Mitigate Hallucination in Multimodal Large Reasoning Models via Reasoning-Conditioned Preference Optimization