mlsanityData to 14 Oct 2026

Reasoning About Reasoning: Towards Informed and Reflective Use of LLM Reasoning in HCI

arXiv 2510.22978 ↗First posted 27 Oct 2025cs.HCcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Reasoning is a distinctive human-like characteristic attributed to LLMs in HCI due to their ability to simulate various human-level tasks. However, this work argues that the reasoning behavior of LLMs in HCI is often decontextualized from the underlying mechanics and subjective decisions that condition the emergence and human interpretation of this behavior. Through a systematic survey of 258 CHI papers from 2020-2025 on LLMs, we discuss how HCI hardly perceives LLM reasoning as a product of sociotechnical orchestration and often references it as an object of application. We argue that such abstraction leads to oversimplification of reasoning methodologies from NLP/ML and results in a distortion of LLMs' empirically studied capabilities and (un)known limitations. Finally, drawing on literature from both NLP/ML and HCI, as a constructive step forward, we develop reflection prompts to support HCI practitioners engage with LLM reasoning in an informed and reflective way.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 33 of its 208 references that are in the map. First posted on arXiv 27 Oct 2025.

Builds on

33 of 208 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Reasoning Language Models: A Blueprint20 Jan
  2. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models16 Jan
  3. 2023A Survey of Reasoning with Foundation Models17 Dec
  4. 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models2 Oct
  5. 2022Towards Reasoning in Large Language Models: A Survey20 Dec
  6. 2022Faithful Reasoning Using Large Language Models30 Aug

Other references · 27

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2024OpenAI o1 System Cardcited by 1,244
  5. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  6. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  7. 2022A Survey on In-context Learningcited by 721
  8. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  9. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
  10. 2022Reasoning with Language Model Prompting: A Surveycited by 252
  11. 2024AI Agents That Mattercited by 219
  12. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 202
15 more
  1. 2022Teaching Algorithmic Reasoning via In-context Learningcited by 112
  2. 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoningcited by 110
  3. 2023Large Language Models as Analogical Reasonerscited by 107
  4. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generationcited by 105
  5. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Modelscited by 105
  6. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 103
  7. 2024The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunitiescited by 64
  8. 2023Guiding Language Model Reasoning with Planning Tokenscited by 58
  9. 2025Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMscited by 31
  10. 2025The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learningcited by 30
  11. 2024How Interpretable are Reasoning Explanations from Prompting Large Language Models?cited by 29
  12. 2025When One LLM Drools, Multi-LLM Collaboration Rulescited by 28
  13. 2024MuMath-Code: Combining Tool-Use Large Language Models with Multi-perspective Data Augmentation for Mathematical Reasoningcited by 22
  14. 2024Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulnesscited by 20
  15. 2025Argument-Based Consistency in Toxicity Explanations of LLMscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Thoughts without Thinking: Reconsidering the Explanatory Value of Chain-of-Thought Reasoning in LLMs through Agentic Pipelines
  2. 2024Are LLMs classical or nonmonotonic reasoners? Lessons from generics
  3. 2025Cognitive Foundations for Reasoning and Their Manifestation in LLMs
  4. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes
  5. 2023Studying and improving reasoning in humans and machines
  6. 2023Are LLMs the Master of All Trades? : Exploring Domain-Agnostic Reasoning Skills of LLMs
  7. 2026Reasoning as Pattern Matching: Shared Mechanisms in Human and LLM Everyday Reasoning
  8. 2025Reasoning LLMs are Wandering Solution Explorers