# Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoning

arXiv: https://arxiv.org/abs/2601.14750 · HTML: https://mlsanity.com/paper/2601.14750/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 21 Jan 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Efficient LLM inference and serving / [Visual token pruning in multimodal models](https://mlsanity.com/inference/visual-token-pruning-in-multimodal-models/)

## Builds on (most cited first, 27 of 27)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning](https://mlsanity.com/paper/2503.04697/) (2025)
- [O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning](https://mlsanity.com/paper/2501.12570/) (2025)
- [TokenSkip: Controllable Chain-of-Thought Compression in LLMs](https://mlsanity.com/paper/2502.12067/) (2025)
- [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://mlsanity.com/paper/2502.21074/) (2025)
- [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://mlsanity.com/paper/2311.01460/) (2023)
- [SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs](https://mlsanity.com/paper/2502.12134/) (2025)
- [Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought](https://mlsanity.com/paper/2501.04682/) (2025)
- [LightThinker: Thinking Step-by-Step Compression](https://mlsanity.com/paper/2502.15589/) (2025)
- [Latent Visual Reasoning](https://mlsanity.com/paper/2509.24251/) (2025)
- [Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains](https://mlsanity.com/paper/2505.16552/) (2025)
- [Efficient Reasoning with Hidden Thinking](https://mlsanity.com/paper/2501.19201/) (2025)
- [Glyph: Scaling Context Windows via Visual-Text Compression](https://mlsanity.com/paper/2510.17800/) (2025)
- [LLM Latent Reasoning as Chain of Superposition](https://mlsanity.com/paper/2510.15522/) (2025)
- [Hybrid Latent Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2505.18454/) (2025)
- [R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search](https://mlsanity.com/paper/2505.16838/) (2025)
- [VeriThinker: Learning to Verify Makes Reasoning Model Efficient](https://mlsanity.com/paper/2505.17941/) (2025)
- [From Text to Pixel: Advancing Long-Context Understanding in MLLMs](https://mlsanity.com/paper/2405.14213/) (2024)
- [Deep Thinking by Markov Chain of Continuous Thoughts](https://mlsanity.com/paper/2509.25020/) (2025)
- [PixelWorld: How Far Are We from Perceiving Everything as Pixels?](https://mlsanity.com/paper/2501.19339/) (2025)

## Built on by (most cited first, 20 of 20)

- [The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook](https://mlsanity.com/paper/2604.02029/) (2026)
- [Imagination Helps Visual Reasoning, But Not Yet in Latent Space](https://mlsanity.com/paper/2602.22766/) (2026)
- [Latent-GRPO: Group Relative Policy Optimization for Latent Reasoning](https://mlsanity.com/paper/2604.27998/) (2026)
- [The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems](https://mlsanity.com/paper/2602.15382/) (2026)
- [VTC-R1: Vision-Text Compression for Efficient Long-Context Reasoning](https://mlsanity.com/paper/2601.22069/) (2026)
- [UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs](https://mlsanity.com/paper/2605.11856/) (2026)
- [Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2605.12374/) (2026)
- [Thinking with Gaze: Sequential Eye-Tracking as Visual Reasoning Supervision for Medical VLMs](https://mlsanity.com/paper/2603.06697/) (2026)
- [Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains](https://mlsanity.com/paper/2605.25745/) (2026)
- [Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression](https://mlsanity.com/paper/2606.29712/) (2026)
- [Decoupling semantics from vision: A framework for faithful visual-text compression evaluation](https://mlsanity.com/paper/2608.01848/) (2026)
- [Latent Action Control for Reasoning-Guided Unified Image Generation](https://mlsanity.com/paper/2605.16961/) (2026)
- [VTI-CoT: Visual-Textual Interleaved Chain of Thought for Video Reasoning](https://mlsanity.com/paper/2606.05736/) (2026)
- [Assessing Adversarial Robustness of Latent Reasoning Models](https://mlsanity.com/paper/2609.22228/) (2026)
- [Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text](https://mlsanity.com/paper/2606.09585/) (2026)
- [VideoLatent: Video-Language Learning via Latent Self-Forcing](https://mlsanity.com/paper/2606.22870/) (2026)
- [SLPO: Scaling Latent Reasoning via a Surrogate Policy](https://mlsanity.com/paper/2607.19691/) (2026)
- [Can Coding Agents Solve Repository-Level Issues with Rendered Code? An Exploratory Study of Visual Representations](https://mlsanity.com/paper/2608.09268/) (2026)
- [MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrieval](https://mlsanity.com/paper/2609.21018/) (2026)
- [Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence](https://mlsanity.com/paper/2609.34563/) (2026)

## Abstract (arXiv)

Chain-of-Thought (CoT) prompting has achieved remarkable success in unlocking the reasoning capabilities of Large Language Models (LLMs). Although CoT prompting enhances reasoning, its verbosity imposes substantial computational overhead. Recent works often focus exclusively on outcome alignment and lack supervision on the intermediate reasoning process. These deficiencies obscure the analyzability of the latent reasoning chain. To address these challenges, we introduce Render-of-Thought (RoT), the first framework to reify the reasoning chain by rendering textual steps into images, making the latent rationale explicit and traceable. Specifically, we leverage the vision encoders of existing Vision Language Models (VLMs) as semantic anchors to align the vision embeddings with the textual space. This design ensures plug-and-play implementation without incurring additional pre-training overhead. Extensive experiments on mathematical and logical reasoning benchmarks demonstrate that our method achieves 3-4x token compression and substantial inference acceleration compared to explicit CoT. Furthermore, it maintains competitive performance against other methods, validating the feasibility of this paradigm. Our code is available at https://github.com/TencentBAC/RoT