mlsanityData to 14 Oct 2026

Evaluating Language-Model Agents on Realistic Autonomous Tasks

arXiv 2312.11671 ↗First posted 18 Dec 2023cs.CL cs.AI cs.LGcited by 96

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

In this report, we explore the ability of language model agents to acquire resources, create copies of themselves, and adapt to novel challenges they encounter in the wild. We refer to this cluster of capabilities as "autonomous replication and adaptation" or ARA. We believe that systems capable of ARA could have wide-reaching and hard-to-anticipate consequences, and that measuring and forecasting ARA may be useful for informing measures around security, monitoring, and alignment. Additionally, once a system is capable of ARA, placing bounds on a system's capabilities may become significantly more difficult. We construct four simple example agents that combine language models with tools that allow them to take actions in the world. We then evaluate these agents on 12 tasks relevant to ARA. We find that these language model agents can only complete the easiest tasks from this list, although they make some progress on the more challenging tasks. Unfortunately, these evaluations are not adequate to rule out the possibility that near-future agents will be capable of ARA. In particular, we do not think that these evaluations provide good assurance that the ``next generation'' of language models (e.g. 100x effective compute scaleup on existing models) will not yield agents capable of ARA, unless intermediate evaluations are performed during pretraining. Relatedly, we expect that fine-tuning of the existing models could produce substantially more competent agents, even if the fine-tuning is not directly targeted at ARA.

Key facts. Cited by 96 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 14 of its 23 references that are in the map. First posted on arXiv 18 Dec 2023.

Builds on

14 of 23 references.

References · 14

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  8. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  9. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  10. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  11. 2021Measuring Coding Challenge Competence With APPScited by 902
  12. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
2 more
  1. 2023Model evaluation for extreme riskscited by 177
  2. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 155

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 43; 2025: 26; 2026: 27.

96 papers.

Sort:

By area:

  1. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  2. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  3. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  4. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
  5. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
  6. 2024Frontier Models are Capable of In-context Schemingcited by 287
  7. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
  8. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agentscited by 145
  9. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  10. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 125
  11. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  12. 2025Establishing Best Practices for Building Rigorous Agentic Benchmarkscited by 102
38 more
  1. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agentscited by 95
  2. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 95
  3. 2024Visibility into AI Agentscited by 79
  4. 2025SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agentscited by 71
  5. 2024Large Language Model Safety: A Holistic Surveycited by 59
  6. 2025Contextual Agent Security: A Policy for Every Purposecited by 59
  7. 2024Determinants of LLM-assisted Decision-Makingcited by 56
  8. 2024MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Controlcited by 51
  9. 2024Stress-Testing Capability Elicitation With Password-Locked Modelscited by 46
  10. 2024Towards evaluations-based safety cases for AI schemingcited by 33
  11. 2025Robotouille: An Asynchronous Planning Benchmark for LLM Agentscited by 32
  12. 2024Exploring Autonomous Agents through the Lens of Large Language Models: A Reviewcited by 31
  13. 2025Reliable Weak-to-Strong Monitoring of LLM Agentscited by 31
  14. 2024When LLMs Go Online: The Emerging Threat of Web-Enabled LLMscited by 24
  15. 2024The Shutdown Problem: An AI Engineering Puzzle for Decision Theoristscited by 22
  16. 2025AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agentscited by 22
  17. 2024AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracycited by 21
  18. 2025AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Modelscited by 21
  19. 2024Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Modelscited by 20
  20. 2024IDs for AI Systemscited by 19
  21. 2024Frontier AI systems have surpassed the self-replicating red linecited by 17
  22. 2024Can LLMs make trade-offs involving stipulated pain and pleasure states?cited by 16
  23. 2024Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Modelscited by 15
  24. 2025RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agentscited by 15
  25. 2025Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correctioncited by 14
  26. 2026Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarioscited by 14
  27. 2024Evaluating AI Evaluation: Perils and Prospectscited by 11
  28. 2025Paradigms of AI Evaluation: Mapping Goals, Methodologies and Culturecited by 11
  29. 2025Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agentscited by 11
  30. 2025Measuring AI agent autonomy: Towards a scalable approach with code inspectioncited by 10
  31. 2024OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safetycited by 9
  32. 2025Discovering Forbidden Topics in Language Modelscited by 9
  33. 2025Dive into the Agent Matrix: A Realistic Evaluation of Self-Replication Risk in LLM Agentscited by 7
  34. 2024Conversational Complexity for Assessing Risk in Large Language Modelscited by 6
  35. 2025LM Agents May Fail to Act on Their Own Risk Knowledgecited by 6
  36. 2024Towards Unified Alignment Between Agents, Humans, and Environmentcited by 5
  37. 2024Securing the Future of GenAI: Policy and Technologycited by 4
  38. 2024Towards a Realistic Long-Term Benchmark for Open-Web Research Agentscited by 4

50 most cited of 96 All 96 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026PACE: Two-Timescale Self-Evolution for Small Language Model Agents
  2. 2026AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
  3. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence
  4. 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward
  5. 2025Small Language Models are the Future of Agentic AI
  6. 2025Towards a Science of Scaling Agent Systems
  7. 2026From Cognitive Architectures to Language Agents: A Mechanism-Level Review of Lineage, Convergence, and Migration Gaps
  8. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges