# ContextCov: Deriving and Enforcing Executable Constraints from Agent Instruction Files

arXiv: https://arxiv.org/abs/2603.00822 · HTML: https://mlsanity.com/paper/2603.00822/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 68 references that are in the map.
- First posted on arXiv 28 Feb 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 18 of 18)

- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations](https://mlsanity.com/paper/2310.06387/) (2023)
- [Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code](https://mlsanity.com/paper/2404.00971/) (2024)
- [AgentRx: Diagnosing AI Agent Failures from Execution Trajectories](https://mlsanity.com/paper/2602.02475/) (2026)
- [Getafix: Learning to Fix Bugs Automatically](https://mlsanity.com/paper/1902.06111/) (2019)
- [Control Illusion: The Failure of Instruction Hierarchies in Large Language Models](https://mlsanity.com/paper/2502.15851/) (2025)
- [Learning to Update Natural Language Comments Based on Code Changes](https://mlsanity.com/paper/2004.12169/) (2020)
- [Agent READMEs: An Empirical Study of Context Files for Agentic Coding](https://mlsanity.com/paper/2511.12884/) (2025)
- [Categorizing the Content of GitHub README Files](https://mlsanity.com/paper/1802.06997/) (2018)
- [Code Search: A Survey of Techniques for Finding Code](https://mlsanity.com/paper/2204.02765/) (2022)
- [On the Effectiveness of LLM-as-a-judge for Code Generation and Summarization](https://mlsanity.com/paper/2507.16587/) (2025)
- [Interpreting and Mitigating Hallucination in MLLMs through Multi-agent Debate](https://mlsanity.com/paper/2407.20505/) (2024)
- [LLMs can be easily Confused by Instructional Distractions](https://mlsanity.com/paper/2502.04362/) (2025)
- [A Systematic Literature Review of Code Hallucinations in LLMs: Characterization, Mitigation Methods, Challenges, and Future Directions for Reliable AI](https://mlsanity.com/paper/2511.00776/) (2025)
- [The Introduction of README and CONTRIBUTING Files in Open Source Software Development](https://mlsanity.com/paper/2502.18440/) (2025)
- [Long-Term Evaluation of Technical Debt in Open-Source Software](https://mlsanity.com/paper/2007.13422/) (2020)

## Built on by (most cited first, 6 of 6)

- [Natural-Language Agent Harnesses](https://mlsanity.com/paper/2603.25723/) (2026)
- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)
- [Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents](https://mlsanity.com/paper/2606.13174/) (2026)
- [OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents](https://mlsanity.com/paper/2608.05013/) (2026)
- [Learning Correct Behavior from Examples: Validating Sequential Execution in Autonomous Agents](https://mlsanity.com/paper/2605.03159/) (2026)
- [Toward Measuring Structural Drift in LLM Communication Loops](https://mlsanity.com/paper/2604.13061/) (2026)

## Abstract (arXiv)

As Large Language Model (LLM) agents increasingly execute complex, autonomous software engineering tasks, developers rely on natural language instruction files such as AGENTS.md to express project-specific coding conventions, tooling restrictions, and architectural boundaries. However, because these instructions remain passive text, agents frequently violate documented constraints due to context window saturation or conflicting local context. In autonomous settings without real-time human supervision, such violations rapidly compound into technical debt. To ground autonomous agents in repository constraints, we introduce ContextCov, a framework that transforms passive natural language instructions into executable guardrails. Unlike prompt-only or reflection-only compliance approaches, ContextCov compiles documented constraints into three complementary checks: static AST queries for code patterns, runtime shell shims that intercept prohibited commands, and architectural validators that enforce structural rules. Acting as an automated, continuous reviewer, ContextCov intercepts agent actions and returns immediate, reproducible violation traces, enabling self-correction before non-compliant changes are finalized. We evaluate ContextCov on SWE-bench Lite (12 repositories, 300 tasks). Compared to prompt-only and LLM reflection baselines, ContextCov achieves 88.3% constraint compliance (vs. 67.0% and 50.3%) with 3.4x lower feedback cost, while maintaining functional correctness. The source code and evaluation results are available at https://github.com/reSHARMA/ContextCov.