Falsifiable Release Gates for Self-Improving Systems: Standing Invariants at Scale
On the map: Synthetic data and self-improvement for LLMs → Self-evolving LLM agents
also in 1 more
Abstract (arXiv)
Safety claims for self-improving agent runtimes are almost always self-graded: a policy file, a guardrail, a promise in a README. We describe falsifiable release gates, a methodology in which every new capability must pass a pre-declared, machine-checkable acceptance suite before it ships, while a fixed set of standing invariants is preserved across every gate. We instantiate it in Antahkarana, an open runtime, then do what a method paper is only vindicated by: we follow the same runtime as it grows and ask whether the guarantees survive. The safety-critical property, that no action reaches an effector without a capability token minted by a control ring, is machine-checked exhaustively over the reachable states of a bounded model; a deliberately broken model yields the shortest counterexample, so the checker demonstrably has teeth. We then carry the runtime through six further releases. Across every one, the action-safety invariants INV-1 through INV-6 held without a single change, and one release added three capabilities while introducing no new invariant. Under the same teeth discipline, six more machine-checked families were added: memory with provable unlearning, a governed agent, calibrated abstention over a post-quantum record, a harness of many sub-agents, the self-improvement loop itself, and the residency of what it produces. The acceptance suite grew from 122 tests to 563. The load-bearing result sits in the negative space: across more than a doubling of capability, the safety core was neither weakened nor redesigned. The last families are the first on real hardware: gated self-improvement compounds a small model from 20% to 70% accuracy while auto-rejecting a candidate that only inflates confidence, and the whole governed path costs 0.021 ms per request, 0.008% of model inference. We release the runtime, tools, and gate suite; every number reproduces with a single command.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Self-evolving LLM agents, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 16 of its 31 references that are in the map. First posted on arXiv 11 Jul 2026.
Builds on
16 of 31 references.
References · 16
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2016Concrete Problems in AI Safetycited by 1,582
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2019Machine Unlearningcited by 1,020
4 more
- 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
- 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment
- 2026Safety Does Not Compose: Non-Decaying Loop State for Autonomous LLM Agents
- 2026Harness as an Asset: Enforcing Determinism via the Convergent AI Agent Framework (CAAF)
- 2026Containment Verification: AI Safety Guarantees Independent of Alignment
- 2026CausalNav: Reliability-Certified Causal World Models for Control under Physical-Parameter Shift
- 2024Quantifying stability of non-power-seeking in artificial agents
- 2026ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures
- 2026Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agents