Goal-Directedness is in the Eye of the Beholder
Not on the map. Why?
Abstract (arXiv)
Our ability to predict the behavior of complex agents turns on the attribution of goals. Probing for goal-directed behavior comes in two flavors: Behavioral and mechanistic. The former proposes that goal-directedness can be estimated through behavioral observation, whereas the latter attempts to probe for goals in internal model states. We work through the assumptions behind both approaches, identifying technical and conceptual problems that arise from formalizing goals in agent systems. We arrive at the perhaps surprising position that goal-directedness cannot be measured objectively. We outline new directions for modeling goal-directedness as an emergent property of dynamic, multi-agent systems.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 12 of its 34 references that are in the map. First posted on arXiv 18 Aug 2025.
Builds on
12 of 34 references.
References · 12
- 2016The Mythos of Model Interpretabilitycited by 721
- 2023Function Vectors in Large Language Modelscited by 308
- 2024Frontier Models are Capable of In-context Schemingcited by 287
- 2025Multi-Agent Risks from Advanced AIcited by 163
- 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 125
- 2021Agent Incentives: A Causal Perspectivecited by 51
- 2018Agents and Devices: A Relative Definition of Agencycited by 16
- 2025Evaluating the Goal-Directedness of Large Language Modelscited by 16
- 2019Modeling AGI Safety Frameworks with Causal Influence Diagramscited by 8
- 2024Towards Measuring Goal-Directedness in AI Systemscited by 7
- 2024Measuring Goal-Directednesscited by 7
- 2023On Imperfect Recall in Multi-Agent Influence Diagramscited by 4
Built on by
Citing papers per year: 2026: 3.
3 papers.