# Tracking Capabilities for Safer Agents

arXiv: https://arxiv.org/abs/2603.00991 · HTML: https://mlsanity.com/paper/2603.00991/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 18 of its 81 references that are in the map.
- First posted on arXiv 1 Mar 2026.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 18 of 18)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024)
- [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://mlsanity.com/paper/2401.10019/) (2024)
- [The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections](https://mlsanity.com/paper/2510.09023/) (2025)
- [GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning](https://mlsanity.com/paper/2406.09187/) (2024)
- [MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits](https://mlsanity.com/paper/2504.03767/) (2025)
- [SAGA: A Security Architecture for Governing AI Agentic Systems](https://mlsanity.com/paper/2504.21034/) (2025)
- [Towards Verifiably Safe Tool Use for LLM Agents](https://mlsanity.com/paper/2601.08012/) (2026)
- [GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applications](https://mlsanity.com/paper/2404.06921/) (2024)
- [Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environment](https://mlsanity.com/paper/2508.04865/) (2025)
- [What's in the Box: Ergonomic and Expressive Capture Tracking over Generic Data Structures (Extended Version)](https://mlsanity.com/paper/2509.07609/) (2025)
- [Designing with Static Capabilities and Effects: Use, Mention, and Invariants](https://mlsanity.com/paper/2005.11444/) (2020)

## Built on by (most cited first, 4 of 4)

- [Agent libOS: A Runtime Substrate for Capability-Controlled Self-Evolving LLM Agents](https://mlsanity.com/paper/2606.03895/) (2026)
- [Explanation-Bound Tool Execution for AI Agents: Server-Verified Action Claims Without Trusting Model Rationales](https://mlsanity.com/paper/2607.25364/) (2026)
- [Formal Semantics for Agentic Tool Protocols: A Process Calculus Approach](https://mlsanity.com/paper/2603.24747/) (2026)
- [Authority Is Not a String: A Capability-Scoped Harness for Prompt-Injection-Resistant Coding Agents](https://mlsanity.com/paper/2609.08371/) (2026)

## Abstract (arXiv)

AI agents that interact with the real world through tool calls pose fundamental safety challenges: agents might leak private information, cause unintended side effects, or be manipulated through prompt injection. To address these challenges, we propose to put the agent in a programming-language-based "safety harness": instead of calling tools directly, agents express their intentions as code in a capability-safe language: Scala 3 with capture checking. Capabilities are program variables that regulate access to effects and resources of interest. Scala's type system tracks capabilities statically, providing fine-grained control over what an agent can do. In particular, it enables local purity, the ability to enforce that sub-computations are side-effect-free, preventing information leakage when agents process classified data. We demonstrate that extensible agent safety harnesses can be built by leveraging a strong type system with tracked capabilities. Our experiments show that agents can generate capability-safe code with no significant loss in task performance, while the type system reliably prevents unsafe behaviors such as information leakage and malicious side effects.