# A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents

arXiv: https://arxiv.org/abs/2602.08964 · HTML: https://mlsanity.com/paper/2602.08964/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 31)

- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks](https://mlsanity.com/paper/2306.13831/) (2023)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Language Models Represent Space and Time](https://mlsanity.com/paper/2310.02207/) (2023)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [Risks from Learned Optimization in Advanced Machine Learning Systems](https://mlsanity.com/paper/1906.01820/) (2019)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [Inverse Reward Design](https://mlsanity.com/paper/1711.02827/) (2017)
- [Implicit Representations of Meaning in Neural Language Models](https://mlsanity.com/paper/2106.00737/) (2021)
- [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024)
- [Future Lens: Anticipating Subsequent Tokens from a Single Hidden State](https://mlsanity.com/paper/2311.04897/) (2023)
- [Auditing language models for hidden objectives](https://mlsanity.com/paper/2503.10965/) (2025)
- [Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs](https://mlsanity.com/paper/2502.08640/) (2025)
- [CUTE: Measuring LLMs' Understanding of Their Tokens](https://mlsanity.com/paper/2409.15452/) (2024)
- [The Reasons that Agents Act: Intention and Instrumental Goals](https://mlsanity.com/paper/2402.07221/) (2024)
- [Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2406.16033/) (2024)
- [AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents](https://mlsanity.com/paper/2506.04018/) (2025)
- [Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language](https://mlsanity.com/paper/2507.03409/) (2025)
- [Emergent Response Planning in LLMs](https://mlsanity.com/paper/2502.06258/) (2025)
- [Planning in a recurrent neural network that plays Sokoban](https://mlsanity.com/paper/2407.15421/) (2024)
- [Evaluating the Goal-Directedness of Large Language Models](https://mlsanity.com/paper/2504.11844/) (2025)
- [Interpreting Emergent Planning in Model-Free Reinforcement Learning](https://mlsanity.com/paper/2504.01871/) (2025)
- [Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectives](https://mlsanity.com/paper/2511.06626/) (2025)
- [Auditing Games for Sandbagging](https://mlsanity.com/paper/2512.07810/) (2025)
- [Propositional Interpretability in Artificial Intelligence](https://mlsanity.com/paper/2501.15740/) (2025)
- [The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models](https://mlsanity.com/paper/2505.14172/) (2025)
- [Towards Measuring Goal-Directedness in AI Systems](https://mlsanity.com/paper/2410.04683/) (2024)
- [The Limits of Predicting Agents from Behaviour](https://mlsanity.com/paper/2506.02923/) (2025)

## Built on by (most cited first, 1 of 1)

- [The belief-action gap in language model agents](https://mlsanity.com/paper/2610.17444/) (2026)

## Abstract (arXiv)

Understanding an agent's goals helps explain and predict its behaviour, yet there is no established methodology for reliably attributing goals to agentic systems. We propose a framework for evaluating goal-directedness that integrates behavioural evaluation with interpretability-based analyses of models' internal representations. As a case study, we examine an LLM agent navigating a 2D grid world towards a goal state. Behaviourally, we evaluate the agent against optimal policies across varying grid sizes, obstacle densities, and goal structures, finding that performance scales with task difficulty while remaining robust to difficulty-preserving transformations and multi-goal structures. We then use probing methods to decode internal representations of the environment and multi-step action plans. We find that the LLM agent non-linearly encodes a coarse spatial map, preserving approximate task-relevant cues about its position and the goal location; that its actions are broadly consistent with these internal representations; and that reasoning reorganises them, shifting from spatial cues towards immediate action selection. Our findings support the view that introspective examination is required beyond behavioural evaluations to characterise how agents represent and pursue their objectives.