# Multi-modal Latent Space Learning for Chain-of-Thought Reasoning in Language Models

arXiv: https://arxiv.org/abs/2312.08762 · HTML: https://mlsanity.com/paper/2312.08762/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 3 of its 38 references that are in the map.
- First posted on arXiv 14 Dec 2023.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 3 of 3)

- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning](https://mlsanity.com/paper/2110.13214/) (2021)
- [Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation](https://mlsanity.com/paper/2105.14462/) (2021)

## Built on by (most cited first, 25 of 25)

- [DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning](https://mlsanity.com/paper/2505.14362/) (2025)
- [Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2503.12605/) (2025)
- [M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought](https://mlsanity.com/paper/2405.16473/) (2024)
- [VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models](https://mlsanity.com/paper/2511.11007/) (2025)
- [Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought](https://mlsanity.com/paper/2505.15510/) (2025)
- [Cantor: Inspiring Multimodal Chain-of-Thought of MLLM](https://mlsanity.com/paper/2404.16033/) (2024)
- [Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2412.11077/) (2024)
- [Stop Reasoning! When Multimodal LLM with Chain-of-Thought Reasoning Meets Adversarial Image](https://mlsanity.com/paper/2402.14899/) (2024)
- [Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuning](https://mlsanity.com/paper/2502.09673/) (2025)
- [ML-Mamba: Efficient Multi-Modal Large Language Model Utilizing Mamba-2](https://mlsanity.com/paper/2407.19832/) (2024)
- [Soft-Prompting with Graph-of-Thought for Multi-modal Representation Learning](https://mlsanity.com/paper/2404.04538/) (2024)
- [PARTONOMY: Large Multimodal Models with Part-Level Visual Understanding](https://mlsanity.com/paper/2505.20759/) (2025)
- [Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identification](https://mlsanity.com/paper/2501.02434/) (2025)
- [Segment First or Comprehend First? Explore the Limit of Unsupervised Word Segmentation with Large Language Models](https://mlsanity.com/paper/2505.19631/) (2025)
- [PASG: A Closed-Loop Framework for Automated Geometric Primitive Extraction and Semantic Anchoring in Robotic Manipulation](https://mlsanity.com/paper/2508.05976/) (2025)
- [From Parameters to Performance: A Data-Driven Study on LLM Structure and Development](https://mlsanity.com/paper/2509.18136/) (2025)
- [MIND: Multi-rationale INtegrated Discriminative Reasoning Framework for Multi-modal Large Models](https://mlsanity.com/paper/2512.05530/) (2025)
- [Explicit Inductive Inference using Large Language Models](https://mlsanity.com/paper/2408.14467/) (2024)
- [Unify and Triumph: Polyglot, Diverse, and Self-Consistent Generation of Unit Tests with LLMs](https://mlsanity.com/paper/2503.16144/) (2025)
- [MAVL: A Multilingual Audio-Video Lyrics Dataset for Animated Song Translation](https://mlsanity.com/paper/2505.18614/) (2025)
- [Rationale-Enhanced Decoding for Multi-modal Chain-of-Thought](https://mlsanity.com/paper/2507.07685/) (2025)
- [Correcting What You Cannot See: Credit Assignment for Perception Distillation in Multimodal Reasoners](https://mlsanity.com/paper/2607.28336/) (2026)
- [Efficient Multimodal Generative Recommendation with Latent Narrative Reasoning](https://mlsanity.com/paper/2609.16070/) (2026)
- [Uni-LaDiR: Latent Diffusion Unifies Multimodal Reasoning](https://mlsanity.com/paper/2609.19878/) (2026)
- [From Stages to States: Unified Iterative Reasoning for MLLMs](https://mlsanity.com/paper/2610.15620/) (2026)

## Abstract (arXiv)

Chain-of-thought (CoT) reasoning has exhibited impressive performance in language models for solving complex tasks and answering questions. However, many real-world questions require multi-modal information, such as text and images. Previous research on multi-modal CoT has primarily focused on extracting fixed image features from off-the-shelf vision models and then fusing them with text using attention mechanisms. This approach has limitations because these vision models were not designed for complex reasoning tasks and do not align well with language thoughts. To overcome this limitation, we introduce a novel approach for multi-modal CoT reasoning that utilizes latent space learning via diffusion processes to generate effective image features that align with language thoughts. Our method fuses image features and text representations at a deep level and improves the complex reasoning ability of multi-modal CoT. We demonstrate the efficacy of our proposed method on multi-modal ScienceQA and machine translation benchmarks, achieving state-of-the-art performance on ScienceQA. Overall, our approach offers a more robust and effective solution for multi-modal reasoning in language models, enhancing their ability to tackle complex real-world problems.