mlsanityData to 7 Oct 2026

Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models

arXiv 2503.22165 ↗First posted 28 Mar 2025cs.LGcited by 25

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Numerous applications of large language models (LLMs) rely on their ability to perform step-by-step reasoning. However, the reasoning behavior of LLMs remains poorly understood, posing challenges to research, development, and safety. To address this gap, we introduce landscape of thoughts (LoT), the first landscape visualization tool to inspect the reasoning trajectories with certain reasoning methods on any multi-choice dataset. We represent the textual states in a trajectory as numerical features that quantify the states' distances to the answer choices. These features are then visualized in two-dimensional plots using t-SNE. Qualitative and quantitative analysis with the landscape of thoughts effectively distinguishes between strong and weak models, correct and incorrect answers, as well as different reasoning tasks. It also uncovers undesirable reasoning patterns, such as low consistency and high uncertainty. Additionally, users can adapt LoT to a model that predicts the property they observe. We showcase this advantage by adapting LoT to a lightweight verifier that evaluates the correctness of trajectories. Empirically, this verifier boosts the reasoning accuracy and the test-time scaling effect. The code is publicly available at: https://github.com/tmlr-group/landscape-of-thoughts.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 24 of its 89 references that are in the map. First posted on arXiv 28 Mar 2025.

Builds on

24 of 89 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024DeepSeek-V3 Technical Report27 Dec
  2. 2023Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners24 May
  3. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning19 May

Other references · 21

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  8. 2024OpenAI o1 System Cardcited by 1,220
  9. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  10. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  11. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  12. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 152
9 more
  1. 2024Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Lawscited by 137
  2. 2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tangocited by 128
  3. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 97
  4. 2022Complementary Explanations for Effective In-Context Learningcited by 71
  5. 2023Large Language Models can Learn Rulescited by 66
  6. 2019TriMap: Large-scale Dimensionality Reduction Using Tripletscited by 59
  7. 2024Understanding Chain-of-Thought in LLMs through Information Theorycited by 36
  8. 2023Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributionscited by 32
  9. 2025DreamDDP: Accelerating Data Parallel Distributed LLM Training with Layer-wise Scheduled Partial Synchronizationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 11.

25 papers.

Sort:

By area:

  1. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoningcited by 43
  2. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearningcited by 39
  3. 2025From Debate to Equilibrium: Belief-Driven Multi-Agent LLM Reasoning via Bayesian Nash Equilibriumcited by 24
  4. 2025From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?cited by 21
  5. 2025Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Modelscited by 20
  6. 2026LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signalscited by 18
  7. 2025LLM Unlearning with LLM Beliefscited by 17
  8. 2026Explainable LLM Unlearning Through Reasoningcited by 7
  9. 2025ReJump: A Tree-Jump Representation for Analyzing and Improving LLM Reasoningcited by 5
  10. 2026Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoningcited by 5
  11. 2025KG-TRACES: Enhancing Large Language Models with Knowledge Graph-constrained Trajectory Reasoning and Attribution Supervisioncited by 4
  12. 2025CyclicReflex: Improving Reasoning Models via Cyclical Reflection Token Schedulingcited by 4
13 more
  1. 2025Towards Understanding Valuable Preference Data for Large Language Model Alignmentcited by 4
  2. 2025ReTrace: Interactive Visualizations for Reasoning Traces of Large Reasoning Modelscited by 3
  3. 2025EPiC: Towards Lossless Speedup for Reasoning Training through Edge-Preserving CoT Condensationcited by 2
  4. 2026SliceGraph: Mapping Process Isomers in Multi-Run Chain-of-Thought Reasoningcited by 2
  5. 2025The Universal Landscape of Human Reasoningcited by 1
  6. 2025Understanding Chain-of-Thought in Large Language Models via Topological Data Analysiscited by 1
  7. 2026When the Chain Breaks: Interactive Diagnosis of LLM Chain-of-Thought Reasoning Errorscited by 1
  8. 2026Rate or Fate? RLV^varepsilonR: Reinforcement Learning with Verifiable Noisy Rewardscited by 0
  9. 2026Exploring LLM Features in Predictive Process Monitoring for Small-Scale Event-Logscited by 0
  10. 2026The Easy, the Hard, and the Learnable: Confidence and Difficulty-Adaptive Policy Optimization for LLM Reasoningcited by 0
  11. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0
  12. 2026Is MoE Routing a Huffman Code? Discovering the Frequency-Diversity Law in Chain-of-Thoughtcited by 0
  13. 2026From Concentration to Differentiation and Back: Routing Effective Rank in MoE Reasoning Cohortscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Graphionale: How Graph Visualizations of LLM Rationales Affect Human Decision Making
  2. 2025Beyond the Black Box: Demystifying Multi-Turn LLM Reasoning with VISTA
  3. 2025Thought calibration: Efficient and confident test-time scaling
  4. 2025ReasonGraph: Visualisation of Reasoning Paths
  5. 2026Do LLMs Build World Models From Text? A Multilingual Diagnostic of Spatial Reasoning
  6. 2026Probing the Trajectories of Reasoning Traces in Large Language Models
  7. 2025Interactive Reasoning: Visualizing and Controlling Chain-of-Thought Reasoning in Large Language Models
  8. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance