mlsanityData to 14 Oct 2026

Visibility into AI Agents

arXiv 2401.13138 ↗First posted 23 Jan 2024cs.CY cs.AIcited by 79

Not on the map. Why?

Cited by 41 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Increased delegation of commercial, scientific, governmental, and personal activities to AI agents -- systems capable of pursuing complex goals with limited supervision -- may exacerbate existing societal risks and introduce new risks. Understanding and mitigating these risks involves critically evaluating existing governance structures, revising and adapting these structures where needed, and ensuring accountability of key stakeholders. Information about where, why, how, and by whom certain AI agents are used, which we refer to as visibility, is critical to these objectives. In this paper, we assess three categories of measures to increase visibility into AI agents: agent identifiers, real-time monitoring, and activity logging. For each, we outline potential implementations that vary in intrusiveness and informativeness. We analyze how the measures apply across a spectrum of centralized through decentralized deployment contexts, accounting for various actors in the supply chain including hardware and software service providers. Finally, we discuss the implications of our measures for privacy and concentration of power. Further work into understanding the measures and mitigating their negative impacts can help to build a foundation for the governance of AI agents.

Key facts. Cited by 79 papers in the map (data through 14 Oct 2026). Builds on 79 of its 182 references that are in the map. First posted on arXiv 23 Jan 2024.

Builds on

79 of 182 references.

References · 79

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  8. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  9. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  10. 2018Datasheets for Datasetscited by 1,386
  11. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  12. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
38 more
  1. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  2. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  3. 2018Model Cards for Model Reportingcited by 1,080
  4. 2022A Generalist Agentcited by 808
  5. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
  6. 2023Cognitive Architectures for Language Agentscited by 429
  7. 2022Self-critiquing models for assisting human evaluatorscited by 380
  8. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  9. 2020Algorithmic Recourse: from Counterfactual Explanations to Interventionscited by 250
  10. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
  11. 2022Predictability and Surprise in Large Generative Modelscited by 210
  12. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 182
  13. 2023Model evaluation for extreme riskscited by 177
  14. 2019Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practicescited by 174
  15. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 172
  16. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  17. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 152
  18. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 148
  19. 2022Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reductioncited by 138
  20. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  21. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 138
  22. 2023Auditing large language models: a three-layered approachcited by 137
  23. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 135
  24. 2023Emergent autonomous scientific research capabilities of large language modelscited by 132
  25. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 112
  26. 2023Can Large Language Models Really Improve by Self-critiquing Their Own Plans?cited by 108
  27. 2023Frontier AI Regulation: Managing Emerging Risks to Public Safetycited by 98
  28. 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 96
  29. 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 91
  30. 2022The Fallacy of AI Functionalitycited by 88
  31. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Modelscited by 87
  32. 2022Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governancecited by 67
  33. 2023Preventing Language Models From Hiding Their Reasoningcited by 65
  34. 2023The Gradient of Generative AI Release: Methods and Considerationscited by 63
  35. 2022Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learningcited by 48
  36. 2023Testing Language Model Agents Safely in the Wildcited by 46
  37. 2023Artificial intelligence and biological misuse: Differentiating risks of language models and biological design toolscited by 45
  38. 2021The Flaws of Policies Requiring Human Oversight of Government Algorithmscited by 44

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 29; 2026: 42.

79 papers.

Sort:

By area:

  1. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 169
  2. 2024Beyond Browsing: API-Based Web Agentscited by 78
  3. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 71
  4. 2025Authenticated Delegation and Authorized AI Agentscited by 65
  5. 2024Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMscited by 55
  6. 2025Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directionscited by 48
  7. 2024AgentOps: Enabling Observability of LLM Agentscited by 46
  8. 2025Infrastructure for AI Agentscited by 41
  9. 2024Cocoa: Co-Planning and Co-Execution with AI Agentscited by 39
  10. 2025Internet of Agents: Fundamentals, Applications, and Challengescited by 31
  11. 2025Characterizing AI Agents for Alignment and Governancecited by 30
  12. 2026The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systemscited by 30
38 more
  1. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Surveycited by 28
  2. 2025Monitoring Decomposition Attacks in LLMs with Lightweight Sequential Monitorscited by 25
  3. 2024IDs for AI Systemscited by 19
  4. 2025The AI Agent Indexcited by 15
  5. 2026Auditable Agentscited by 15
  6. 2025Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspectivecited by 13
  7. 2025Distributional AGI Safetycited by 12
  8. 2025Episodic memory in AI agents poses risks that should be studied and mitigatedcited by 11
  9. 2024Societal Adaptation to Advanced AIcited by 10
  10. 2025Measuring AI agent autonomy: Towards a scalable approach with code inspectioncited by 10
  11. 2025From Conversation to Orchestration: HCI Challenges and Opportunities in Interactive Multi-Agentic Systemscited by 9
  12. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 9
  13. 2024The Role of Governments in Increasing Interconnected Post-Deployment Monitoring of AIcited by 7
  14. 2026Training Agents to Self-Report Misbehaviorcited by 6
  15. 2026Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agentscited by 6
  16. 2025Agent Identity Evals: Measuring Agentic Identitycited by 5
  17. 2026TRACE: A Two-Channel Robust Attribution Watermark via Complementary Embeddings for LLM-Agent Trajectoriescited by 5
  18. 2025Reproducibility Study of Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiationcited by 4
  19. 2025Misalignment or misuse? The AGI alignment tradeoffcited by 4
  20. 2025LumiMAS: A Comprehensive Framework for Real-Time Monitoring and Enhanced Observability in Multi-Agent Systemscited by 4
  21. 2026Dissociative Identity: Language Model Agents Lack Grounding for Reputation Mechanismscited by 4
  22. 2024Safeguarding AI Agents: Developing and Analyzing Safety Architecturescited by 3
  23. 2025Forecasting Open-Weight AI Model Growth on HuggingFacecited by 3
  24. 2025On the Regulatory Potential of User Interfaces for AI Agent Governancecited by 3
  25. 2026Agent Benchmarks Fail Public Sector Requirementscited by 3
  26. 2026Does My Chatbot Have an Agenda? Understanding Human and AI Agency in Human-Human-like Chatbot Interactioncited by 3
  27. 2026Formal Methods Meet LLMs: Auditing, Monitoring, and Intervention for Compliance of Advanced AI Systemscited by 3
  28. 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 3
  29. 2026Plans They Abandon, Reports They Author: The Narrative Layer of Autonomous Agentscited by 3
  30. 2025The Role of Generative AI in Software Student CollaborAItioncited by 2
  31. 2025Detecting Malicious AI Agents Through Simulated Interactionscited by 2
  32. 2025The Problem of Algorithmic Collisions: Mitigating Unforeseen Risks in a Connected Worldcited by 2
  33. 2026Will the Agent Recuse, and Will It Stop? Measuring LLM-Agent Compliance with In-Band Governance Signals at the Access Door and Mid-Flightcited by 2
  34. 2026Steganalysis of Adaptive Covert Collusion in Tool-Using Agent Populations: A Black-Box, Cross-Principal Approachcited by 2
  35. 2025Dimensional Characterization and Pathway Modeling for Catastrophic AI Riskscited by 1
  36. 2025Toward a Unified Security Framework for AI Agents: Trust, Risk, and Liabilitycited by 1
  37. 2026Helicase: Uncertainty-Guided Supply Chain Knowledge Graph Construction with Autonomous Multi-Agent LLMscited by 1
  38. 2026Toward Pre-Deployment Assurance for Enterprise AI Agents: Ontology-Grounded Simulation and Trust Certificationcited by 1

50 most cited of 79 All 79 (JSON)