mlsanityData to 7 Oct 2026

Technical Report: Evaluating Goal Drift in Language Model Agents

arXiv 2505.02709 ↗First posted 5 May 2025cs.AI cs.LGcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

As language models (LMs) are increasingly deployed as autonomous agents, their robust adherence to human-assigned objectives becomes crucial for safe operation. When these agents operate independently for extended periods without human oversight, even initially well-specified goals may gradually shift. Detecting and measuring goal drift - an agent's tendency to deviate from its original objective over time - presents significant challenges, as goals can shift gradually, causing only subtle behavioral changes. This paper proposes a novel approach to analyzing goal drift in LM agents. In our experiments, agents are first explicitly given a goal through their system prompt, then exposed to competing objectives through environmental pressures. We demonstrate that while the best-performing agent (a scaffolded version of Claude 3.5 Sonnet) maintains nearly perfect goal adherence for more than 100,000 tokens in our most difficult evaluation setting, all evaluated models exhibit some degree of goal drift. We also find that goal drift correlates with models' increasing susceptibility to pattern-matching behaviors as the context length grows.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 28 of its 41 references that are in the map. First posted on arXiv 5 May 2025.

Builds on

28 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents20 Feb

Other references · 27

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  4. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  7. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  8. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  9. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  10. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  11. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  12. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
15 more
  1. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  2. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 360
  3. 2024Alignment faking in large language modelscited by 316
  4. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
  5. 2023An Overview of Catastrophic AI Riskscited by 214
  6. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 153
  7. 2025Measuring AI Ability to Complete Long Software Taskscited by 142
  8. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 132
  9. 2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
  10. 2023Scheming AIs: Will AIs fake alignment during training in order to get power?cited by 90
  11. 2024Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queriescited by 85
  12. 2025LongProc: Benchmarking Long-Context Language Models on Long Procedural Generationcited by 28
  13. 2018Agents and Devices: A Relative Definition of Agencycited by 16
  14. 2025Evaluating the Goal-Directedness of Large Language Modelscited by 15
  15. 2024Towards Measuring Goal-Directedness in AI Systemscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoningcited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2026Inherited Goal Drift: Contextual Pressure Can Undermine Agentic Goals
  2. 2026Agent Drift: Quantifying Behavioral Degradation in Multi-Agent LLM Systems Over Extended Interactions
  3. 2026A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents
  4. 2026When Users Change Their Minds: Measuring and Repairing Intent Drift in LLM Agents
  5. 2026Language Model Goal Selection Differs from Humans' in a Self-Directed Learning Task
  6. 2026Measuring Cross-Task Behavioral Consistency in Language Model Agents
  7. 2025AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents
  8. 2025AgentChangeBench: A Multi-Dimensional Evaluation Framework for Goal-Shift Robustness in Conversational AI