# Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agents

arXiv: https://arxiv.org/abs/2411.03455 · HTML: https://mlsanity.com/paper/2411.03455/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 31 of its 82 references that are in the map.
- First posted on arXiv 5 Nov 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 31)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [Automated Design of Agentic Systems](https://mlsanity.com/paper/2408.08435/) (2024)
- [AutoCodeRover: Autonomous Program Improvement](https://mlsanity.com/paper/2404.05427/) (2024)
- [Efficient Training of Language Models to Fill in the Middle](https://mlsanity.com/paper/2207.14255/) (2022)
- [The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey](https://mlsanity.com/paper/2404.11584/) (2024)
- [DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning](https://mlsanity.com/paper/2402.17453/) (2024)
- [Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse](https://mlsanity.com/paper/2410.21333/) (2024)
- [Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era](https://mlsanity.com/paper/2404.11457/) (2024)
- [Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond](https://mlsanity.com/paper/2306.09841/) (2023)
- [Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces](https://mlsanity.com/paper/2410.09918/) (2024)
- [Language Models for Code Completion: A Practical Evaluation](https://mlsanity.com/paper/2402.16197/) (2024)
- [Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap](https://mlsanity.com/paper/2410.06107/) (2024)
- [Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks](https://mlsanity.com/paper/2403.04814/) (2024)
- [Towards Training Reproducible Deep Learning Models](https://mlsanity.com/paper/2202.02326/) (2022)
- [AgentLens: Visual Analysis for Agent Behaviors in LLM-based Autonomous Systems](https://mlsanity.com/paper/2402.08995/) (2024)
- [ArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Disscusion via Argumentation Schemes](https://mlsanity.com/paper/2403.06294/) (2024)
- [ReAGent: A Model-agnostic Feature Attribution Method for Generative Language Models](https://mlsanity.com/paper/2402.00794/) (2024)
- [PromptExp: Multi-granularity Prompt Explanation of Large Language Models](https://mlsanity.com/paper/2410.13073/) (2024)

## Built on by (most cited first, 8 of 8)

- [Agentic Software Engineering: Foundational Pillars and a Research Roadmap](https://mlsanity.com/paper/2509.06216/) (2025)
- [ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection](https://mlsanity.com/paper/2605.03378/) (2026)
- [AgentStepper: Interactive Debugging of Software Development Agents](https://mlsanity.com/paper/2602.06593/) (2026)
- [AgentTrace: A Structured Logging Framework for Agent System Observability](https://mlsanity.com/paper/2602.10133/) (2026)
- [REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces](https://mlsanity.com/paper/2606.09071/) (2026)
- [Towards Security-Auditable LLM Agents: A Unified Graph Representation](https://mlsanity.com/paper/2605.06812/) (2026)
- [AI-NativeBench: An Open-Source White-Box Agentic Benchmark Suite for AI-Native Systems](https://mlsanity.com/paper/2601.09393/) (2026)
- [Signals: Trajectory Sampling and Triage for Agentic Interactions](https://mlsanity.com/paper/2604.00356/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly integrated into autonomous systems, giving rise to a new class of software known as Agentware, where LLM-powered agents perform complex, open-ended tasks in domains such as software engineering, customer service, and data analysis. However, their high autonomy and opaque reasoning processes pose significant challenges for traditional software observability methods. To address this, we introduce the concept of cognitive observability - the ability to recover and inspect the implicit reasoning behind agent decisions. We present Watson, a general-purpose framework for observing the reasoning processes of fast-thinking LLM agents without altering their behavior. Watson retroactively infers reasoning traces using prompt attribution techniques. We evaluate Watson in both manual debugging and automated correction scenarios across the MMLU benchmark and the AutoCodeRover and OpenHands agents on the SWE-bench-lite dataset. In both static and dynamic settings, Watson surfaces actionable reasoning insights and supports targeted interventions, demonstrating its practical utility for improving transparency and reliability in Agentware systems.