mlsanityData to 7 Oct 2026

Measuring Goal-Directedness

arXiv 2412.04758 ↗First posted 6 Dec 2024cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

We define maximum entropy goal-directedness (MEG), a formal measure of goal-directedness in causal models and Markov decision processes, and give algorithms for computing it. Measuring goal-directedness is important, as it is a critical element of many concerns about harm from AI. It is also of philosophical interest, as goal-directedness is a key aspect of agency. MEG is based on an adaptation of the maximum causal entropy framework used in inverse reinforcement learning. It can measure goal-directedness with respect to a known utility function, a hypothesis class of utility functions, or a set of random variables. We prove that MEG satisfies several desiderata and demonstrate our algorithms with small-scale experiments.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 27 of its 43 references that are in the map. First posted on arXiv 6 Dec 2024.

Builds on

27 of 43 references.

References · 27

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  3. 2017Counterfactual Fairnesscited by 802
  4. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  5. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  6. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  7. 2023Model evaluation for extreme riskscited by 167
  8. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
  9. 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 122
  10. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 121
  11. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 107
  12. 2024The Ethics of Advanced AI Assistantscited by 103
15 more
  1. 2024Robust agents learn causal world modelscited by 81
  2. 2021Agent Incentives: A Causal Perspectivecited by 51
  3. 2023Honesty Is the Best Policy: Defining and Mitigating AI Deceptioncited by 40
  4. 2018Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibilitycited by 32
  5. 2022Counterfactual harmcited by 29
  6. 2022Discovering Agentscited by 28
  7. 2024The Reasons that Agents Act: Intention and Instrumental Goalscited by 23
  8. 2018Agents and Devices: A Relative Definition of Agencycited by 16
  9. 2022A Primer on Maximum Causal Entropy Inverse Reinforcement Learningcited by 13
  10. 2022Interpreting systems as solving POMDPs: a step towards a formal understanding of agencycited by 7
  11. 2024Towards Measuring Goal-Directedness in AI Systemscited by 7
  12. 2021Interpreting Dynamical Systems as Bayesian Reasonerscited by 5
  13. 2023On Imperfect Recall in Multi-Agent Influence Diagramscited by 4
  14. 2023Characterising Decision Theories with Mechanised Causal Graphscited by 3
  15. 2015Formalizing Preference Utilitarianism in Physical World Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025Evaluating the Goal-Directedness of Large Language Modelscited by 15
  2. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 9
  3. 2025Goal-Directedness is in the Eye of the Beholdercited by 3
  4. 2025A Principle of Targeted Intervention for Multi-Agent Reinforcement Learningcited by 3
  5. 2026The Impossibility of Eliciting Latent Knowledgecited by 1
  6. 2026Causal Foundations of Collective Agencycited by 0
  7. 2026A Taxonomy of Cognitive Capability Gaps in Generative and Agentic AIcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Generalized Maximum Causal Entropy for Inverse Reinforcement Learning
  2. 2019Maximum Causal Entropy Specification Inference from Demonstrations
  3. 2021Task-Guided Inverse Reinforcement Learning Under Partial Information
  4. 2022IRL with Partial Observations using the Principle of Uncertain Maximum Entropy
  5. 2023FP-IRL: Fokker--Planck Inverse Reinforcement Learning -- A Physics-Constrained Approach to Markov Decision Processes
  6. 2025Goals and the Structure of Experience
  7. 2019Maximum Likelihood Constraint Inference for Inverse Reinforcement Learning
  8. 2022Weighted Maximum Entropy Inverse Reinforcement Learning