# The AI Agent Index

arXiv: https://arxiv.org/abs/2502.01635 · HTML: https://mlsanity.com/paper/2502.01635/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 62 of its 101 references that are in the map.
- First posted on arXiv 3 Feb 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 30 of 62)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [TravelPlanner: A Benchmark for Real-World Planning with Language Agents](https://mlsanity.com/paper/2402.01622/) (2024)
- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis](https://mlsanity.com/paper/2307.12856/) (2023)
- [TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks](https://mlsanity.com/paper/2412.14161/) (2024)

## Built on by (most cited first, 15 of 15)

- [Characterizing AI Agents for Alignment and Governance](https://mlsanity.com/paper/2504.21848/) (2025)
- [The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems](https://mlsanity.com/paper/2602.17753/) (2026)
- [How Well Does Agent Development Reflect Real-World Work?](https://mlsanity.com/paper/2603.01203/) (2026)
- [Build Agent Advocates, Not Platform Agents](https://mlsanity.com/paper/2505.04345/) (2025)
- [Position: Collaborative Agentic AI Needs Interoperability Across Ecosystems](https://mlsanity.com/paper/2505.21550/) (2025)
- [TableTalk: Scaffolding Spreadsheet Development with a Language Agent](https://mlsanity.com/paper/2502.09787/) (2025)
- [VET Your Agent: Towards Host-Independent Autonomy via Verifiable Execution Traces](https://mlsanity.com/paper/2512.15892/) (2025)
- [On the Regulatory Potential of User Interfaces for AI Agent Governance](https://mlsanity.com/paper/2512.00742/) (2025)
- [Value-Conflict Diagnostics Reveal Widespread Alignment Faking in Language Models](https://mlsanity.com/paper/2604.20995/) (2026)
- [The Problem of Algorithmic Collisions: Mitigating Unforeseen Risks in a Connected World](https://mlsanity.com/paper/2505.20181/) (2025)
- [Psychological and behavioural responses in human-agent vs. human-human interactions: a systematic review and meta-analysis](https://mlsanity.com/paper/2509.21542/) (2025)
- [Agentic AI for Mobile Network RAN Management and Optimization](https://mlsanity.com/paper/2511.02532/) (2025)
- [ClawNet: Human-Symbiotic Agent Network for Cross-User Autonomous Cooperation](https://mlsanity.com/paper/2604.19211/) (2026)
- [Synchronization Dynamics of Heterogeneous, Collaborative Multi-Agent AI Systems](https://mlsanity.com/paper/2508.12314/) (2025)
- [The Agentic Web Requires New Normative Infrastructure](https://mlsanity.com/paper/2606.10711/) (2026)

## Abstract (arXiv)

Leading AI developers and startups are increasingly deploying agentic AI systems that can plan and execute complex tasks with limited human involvement. However, there is currently no structured framework for documenting the technical components, intended uses, and safety features of agentic systems. To fill this gap, we introduce the AI Agent Index, the first public database to document information about currently deployed agentic AI systems. For each system that meets the criteria for inclusion in the index, we document the system's components (e.g., base model, reasoning implementation, tool use), application domains (e.g., computer use, software engineering), and risk management practices (e.g., evaluation results, guardrails), based on publicly available information and correspondence with developers. We find that while developers generally provide ample information regarding the capabilities and applications of agentic systems, they currently provide limited information regarding safety and risk management practices. The AI Agent Index is available online at https://aiagentindex.mit.edu/