# Governed Capability Evolution: Lifecycle-Time Compatibility Checking and Rollback for AI-Component-Based Systems, with a Proof-of-Concept Evaluation on Embodied Agents

arXiv: https://arxiv.org/abs/2604.08059 · HTML: https://mlsanity.com/paper/2604.08059/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 56 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 30 of 30)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails](https://mlsanity.com/paper/2310.10501/) (2023)
- [Accelerating Reinforcement Learning with Learned Skill Priors](https://mlsanity.com/paper/2010.11944/) (2020)
- [AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents](https://mlsanity.com/paper/2503.18666/) (2025)
- [Robot Operating System 2: Design, Architecture, and Uses In The Wild](https://mlsanity.com/paper/2211.07752/) (2022)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [Behavior Trees in Robotics and AI: An Introduction](https://mlsanity.com/paper/1709.00084/) (2017)
- [AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents](https://mlsanity.com/paper/2401.12963/) (2024)
- [TrustAgent: Towards Safe and Trustworthy LLM-based Agents](https://mlsanity.com/paper/2402.01586/) (2024)
- [Skill-based Model-based Reinforcement Learning](https://mlsanity.com/paper/2207.07560/) (2022)
- [Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges](https://mlsanity.com/paper/1905.04223/) (2019)
- [Safety Guardrails for LLM-Enabled Robots](https://mlsanity.com/paper/2503.07885/) (2025)
- [Run Time Assurance for Safety-Critical Systems: An Introduction to Safety Filtering Approaches for Complex Control Systems](https://mlsanity.com/paper/2110.03506/) (2021)
- [MI9: An Integrated Runtime Governance Framework for Agentic AI](https://mlsanity.com/paper/2508.03858/) (2025)
- [Agent Behavioral Contracts: Formal Specification and Runtime Enforcement for Reliable Autonomous AI Agents](https://mlsanity.com/paper/2602.22302/) (2026)
- [Swiss Cheese Model for AI Safety: A Taxonomy and Reference Architecture for Multi-Layered Guardrails of Foundation Model Based Agents](https://mlsanity.com/paper/2408.02205/) (2024)
- [ACRFence: Preventing Semantic Rollback Attacks in Agent Checkpoint-Restore](https://mlsanity.com/paper/2603.20625/) (2026)
- [Formal Specification and Verification of Autonomous Robotic Systems: A Survey](https://mlsanity.com/paper/1807.00048/) (2018)
- [AEROS: A Single-Agent Operating Architecture with Embodied Capability Modules](https://mlsanity.com/paper/2604.07039/) (2026)
- [Putting the Semantics into Semantic Versioning](https://mlsanity.com/paper/2008.07069/) (2020)
- [Automating Staged Rollout with Reinforcement Learning](https://mlsanity.com/paper/2204.02189/) (2022)
- [LITHE: Bridging Best-Effort Python and Real-Time C++ for Hot-Swapping Robotic Control Laws on Commodity Linux](https://mlsanity.com/paper/2603.07442/) (2026)

## Built on by (most cited first, 1 of 1)

- [Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance](https://mlsanity.com/paper/2609.10105/) (2026)

## Abstract (arXiv)

Systems built from versioned AI components need lifecycle-time governance over how a new module version is admitted, monitored, and withdrawn. Established deployment patterns (canary, blue-green, feature flags, MLOps) monitor aggregate service-level signals of stateless services, not the stateful, policy-constrained runtimes that drive AI components. We formulate governed capability evolution as a software-lifecycle problem and ask whether staged upgrade governance prevents unsafe activations where practice-adapted canary and blue-green deployment do not, and at what cost. We propose four compatibility checks (interface, policy, behavioral, recovery) and a seven-stage pipeline (validation, sandbox, shadow, gated activation, monitoring, rollback, audit). A reference prototype is evaluated in a behavioral simulation of an embodied manipulation stack; a PyBullet backend replicates the strategy ordering. Upgrade candidates come from a seeded fault-operator generator, including out-of-taxonomy probes, and thresholds are fixed on a disjoint development set. Across five strategies, five upgrade rounds and 15 seeds, naive upgrade reaches 73.3% final-round success but activates unsafe versions in 74.7% of rounds. Canary and blue-green adaptations keep comparable success yet still admit faulty versions in 61.3% and 64.0% of rounds, because aggregate success hides policy and recovery regressions. Governed upgrade admits no in-taxonomy fault and one out-of-taxonomy probe in 75 rounds, at a 4.6-point success cost (Holm-adjusted p<=0.024 against all three baselines); the full pre-activation stack detects 97.3-100% of in-taxonomy faults across deployment profiles, and rollback restores verified operation in 72.4% of drift scenarios. Bounded-window aggregate-success gating proved insufficient against these fault operators; staged governance closes the gap at a modest, tunable cost.