# Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models arXiv: https://arxiv.org/abs/2503.22165 ยท HTML: https://mlsanity.com/paper/2503.22165/ ## Key facts - Cited by 25 papers in the map (data through 14 Oct 2026). - Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. - Builds on 24 of its 89 references that are in the map. - First posted on arXiv 28 Mar 2025. ## Placed in - LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/) ## Builds on (most cited first, 24 of 24) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021) - [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025) - [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018) - [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024) - [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016) - [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024) - [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023) - [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024) - [Improve Mathematical Reasoning in Language Models by Automated Process Supervision](https://mlsanity.com/paper/2406.06592/) (2024) - [DeepInception: Hypnotize Large Language Model to Be Jailbreaker](https://mlsanity.com/paper/2311.03191/) (2023) - [Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning](https://mlsanity.com/paper/2205.09712/) (2022) - [The Impact of Reasoning Step Length on Large Language Models](https://mlsanity.com/paper/2401.04925/) (2024) - [Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws](https://mlsanity.com/paper/2404.05405/) (2024) - [Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango](https://mlsanity.com/paper/2209.07686/) (2022) - [Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy](https://mlsanity.com/paper/2210.17546/) (2022) - [Complementary Explanations for Effective In-Context Learning](https://mlsanity.com/paper/2211.13892/) (2022) - [Large Language Models can Learn Rules](https://mlsanity.com/paper/2310.07064/) (2023) - [Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners](https://mlsanity.com/paper/2305.14825/) (2023) - [TriMap: Large-scale Dimensionality Reduction Using Triplets](https://mlsanity.com/paper/1910.00204/) (2019) - [Understanding Chain-of-Thought in LLMs through Information Theory](https://mlsanity.com/paper/2411.11984/) (2024) - [Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributions](https://mlsanity.com/paper/2307.13339/) (2023) - [DreamDDP: Accelerating Data Parallel Distributed LLM Training with Layer-wise Scheduled Partial Synchronization](https://mlsanity.com/paper/2502.11058/) (2025) ## Built on by (most cited first, 25 of 25) - [Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning](https://mlsanity.com/paper/2506.08745/) (2025) - [Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning](https://mlsanity.com/paper/2505.11953/) (2025) - [From Debate to Equilibrium: Belief-Driven Multi-Agent LLM Reasoning via Bayesian Nash Equilibrium](https://mlsanity.com/paper/2506.08292/) (2025) - [LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals](https://mlsanity.com/paper/2604.05655/) (2026) - [From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?](https://mlsanity.com/paper/2506.08295/) (2025) - [Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models](https://mlsanity.com/paper/2508.00410/) (2025) - [LLM Unlearning with LLM Beliefs](https://mlsanity.com/paper/2510.19422/) (2025) - [Explainable LLM Unlearning Through Reasoning](https://mlsanity.com/paper/2603.09980/) (2026) - [Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoning](https://mlsanity.com/paper/2603.05900/) (2026) - [CyclicReflex: Improving Reasoning Models via Cyclical Reflection Token Scheduling](https://mlsanity.com/paper/2506.11077/) (2025) - [ReJump: A Tree-Jump Representation for Analyzing and Improving LLM Reasoning](https://mlsanity.com/paper/2512.00831/) (2025) - [KG-TRACES: Enhancing Large Language Models with Knowledge Graph-constrained Trajectory Reasoning and Attribution Supervision](https://mlsanity.com/paper/2506.00783/) (2025) - [Towards Understanding Valuable Preference Data for Large Language Model Alignment](https://mlsanity.com/paper/2510.13212/) (2025) - [ReTrace: Interactive Visualizations for Reasoning Traces of Large Reasoning Models](https://mlsanity.com/paper/2511.11187/) (2025) - [EPiC: Towards Lossless Speedup for Reasoning Training through Edge-Preserving CoT Condensation](https://mlsanity.com/paper/2506.04205/) (2025) - [SliceGraph: Mapping Process Isomers in Multi-Run Chain-of-Thought Reasoning](https://mlsanity.com/paper/2605.14619/) (2026) - [The Easy, the Hard, and the Learnable: Confidence and Difficulty-Adaptive Policy Optimization for LLM Reasoning](https://mlsanity.com/paper/2606.07950/) (2026) - [The Universal Landscape of Human Reasoning](https://mlsanity.com/paper/2510.21623/) (2025) - [Understanding Chain-of-Thought in Large Language Models via Topological Data Analysis](https://mlsanity.com/paper/2512.19135/) (2025) - [Rate or Fate? RLV^varepsilonR: Reinforcement Learning with Verifiable Noisy Rewards](https://mlsanity.com/paper/2601.04411/) (2026) - [When the Chain Breaks: Interactive Diagnosis of LLM Chain-of-Thought Reasoning Errors](https://mlsanity.com/paper/2603.21286/) (2026) - [Exploring LLM Features in Predictive Process Monitoring for Small-Scale Event-Logs](https://mlsanity.com/paper/2601.11468/) (2026) - [The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes](https://mlsanity.com/paper/2606.11470/) (2026) - [Is MoE Routing a Huffman Code? Discovering the Frequency-Diversity Law in Chain-of-Thought](https://mlsanity.com/paper/2607.20427/) (2026) - [From Concentration to Differentiation and Back: Routing Effective Rank in MoE Reasoning Cohorts](https://mlsanity.com/paper/2609.06403/) (2026) ## Abstract (arXiv) Numerous applications of large language models (LLMs) rely on their ability to perform step-by-step reasoning. However, the reasoning behavior of LLMs remains poorly understood, posing challenges to research, development, and safety. To address this gap, we introduce landscape of thoughts (LoT), the first landscape visualization tool to inspect the reasoning trajectories with certain reasoning methods on any multi-choice dataset. We represent the textual states in a trajectory as numerical features that quantify the states' distances to the answer choices. These features are then visualized in two-dimensional plots using t-SNE. Qualitative and quantitative analysis with the landscape of thoughts effectively distinguishes between strong and weak models, correct and incorrect answers, as well as different reasoning tasks. It also uncovers undesirable reasoning patterns, such as low consistency and high uncertainty. Additionally, users can adapt LoT to a model that predicts the property they observe. We showcase this advantage by adapting LoT to a lightweight verifier that evaluates the correctness of trajectories. Empirically, this verifier boosts the reasoning accuracy and the test-time scaling effect. The code is publicly available at: https://github.com/tmlr-group/landscape-of-thoughts.