# Phantom Guardrails: When Self-Improving Agent Harnesses Fix Failures That Never Happened

arXiv: https://arxiv.org/abs/2607.13083 · HTML: https://mlsanity.com/paper/2607.13083/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)
- Synthetic data and self-improvement for LLMs / [Self-evolving LLM agents](https://mlsanity.com/synth/self-evolving-llm-agents/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 13 of 13)

- [Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses](https://mlsanity.com/paper/2604.25850/) (2026)
- [AutoHarness: improving LLM agents by automatically synthesizing a code harness](https://mlsanity.com/paper/2603.03329/) (2026)
- [Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents](https://mlsanity.com/paper/2605.22166/) (2026)
- [SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models](https://mlsanity.com/paper/2601.03555/) (2026)
- [LLM Agents Already Know When to Call Tools -- Even Without Reasoning](https://mlsanity.com/paper/2605.09252/) (2026)
- [SafetyRepro: Configuration-Conditional Rank Instability on Alignment Benchmarks](https://mlsanity.com/paper/2605.25492/) (2026)
- [You Cannot Fix What You Cannot Find! An Investigation of Fault Localization Bias in Benchmarking Automated Program Repair Systems](https://mlsanity.com/paper/1812.07283/) (2018)
- [Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection](https://mlsanity.com/paper/2605.24834/) (2026)
- [Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?](https://mlsanity.com/paper/2508.19827/) (2025)
- [Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation](https://mlsanity.com/paper/2606.15127/) (2026)
- [Reward Auditor: Inference on Reward Modeling Suitability in Real-World Perturbed Scenarios](https://mlsanity.com/paper/2512.00920/) (2025)
- [The Observability Gap: Why Output-Level Human Feedback Fails for LLM Coding Agents](https://mlsanity.com/paper/2603.26942/) (2026)
- [PAFT: Preservation Aware Fine-Tuning for Minimal-Edit Program Repair](https://mlsanity.com/paper/2604.03113/) (2026)

## Built on by (most cited first, 7 of 7)

- [Evidence-Unit Fairness and the Limits of Query-Adaptive Sparse-Dense Fusion in Financial Document Retrieval](https://mlsanity.com/paper/2608.00183/) (2026)
- [Agentic-SQL Revisited: Autonomy-Based Taxonomy and Empirical Benchmark Analysis for LLM Text-to-SQL](https://mlsanity.com/paper/2608.15389/) (2026)
- [VERSE: Verified Self-Evolving Optimizer for Agent Harnesses](https://mlsanity.com/paper/2610.02616/) (2026)
- [A Contract-Centered Architecture for Scalable and Manageable Agentic Runtimes](https://mlsanity.com/paper/2608.27086/) (2026)
- [Bioinfoysis Technical Report](https://mlsanity.com/paper/2609.03871/) (2026)
- [Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts](https://mlsanity.com/paper/2609.08236/) (2026)
- [Harness Evolution Hits a Ceiling: When Weight Training Should Begin](https://mlsanity.com/paper/2610.11655/) (2026)

## Abstract (arXiv)

Self-improving AI agents are designed to learn from their mistakes. We show they can also hallucinate mistakes that never happened. We study this failure mode in automated harness optimization, where an LLM-based proposer edits an agent's scaffold, including prompts, parsers, filters, validators and guardrails, to eliminate observed failures. But this process rarely asks first: was there a real failure to fix? We introduce the Counterfactual Fabrication Lab, a deterministic micro-lab where the correct action is known: do nothing. The lab plants a candidate guardrail for a failure class that provably never occurs, presents only legal episodes, and uses a byte-exact oracle to check every cited violation. The proposer behaves as expected on real violations and abstains on featureless legal input. Yet when the legal input contains a harmless pattern resembling a familiar game rule, it invents a failure: in 15/60 runs, versus 0/60 on featureless input, it enables the nonexistent-rule guardrail and cites a violation the oracle refutes. The effect is structured, not indiscriminate. In single-shot proposals it appears only when three conditions coincide: a rule-shaped pattern, an open-ended rule set and an instruction that presupposes failures. Removing any of these conditions eliminates the fabrication. Because the invented guardrail changes no true outcome and cannot improve an already-perfect suppression score, the phenomenon is neither reward hacking nor over-refusal. It is a phantom guardrail: a fix for a failure that never happened, invisible to suppression-only acceptance. Inside an add-only accept loop it re-enters even without the failure-presupposing instruction, the loop's keep-adding role supplying the demand the instruction supplied in single shot, and once in it stays. We present the Counterfactual Fabrication Lab for measuring fabricated failures in self-improving agent harnesses.