AEGIS: A Backup Reflex for Physical AI
Not on the map. Why?
Abstract (arXiv)
Long-horizon robot manipulation tends to fail gradually: one bad step degrades the state, and the policy spirals into a basin from which it cannot recover. The failure is often visible before it happens. We introduce AEGIS (Activation-probe Early-warning, Gated Inference Switching), a selective escalation method that uses a lightweight probe on a weak policy's frozen activations to detect high-risk steps while there is still time to act. When the probe flags a step, control switches to a stronger separate policy, but only for the steps that need it. On LIBERO-Spatial, AEGIS recovers 10.1% of the trajectories the weak policy alone loses, versus 4.6% for budget-matched blind escalation and 5.1% for a random-trigger placebo. These gains are significant under one-sided exact paired McNemar tests with Holm-Bonferroni adjustment over three pre-registered contrasts: +5.4pp over blind escalation, p=8.5e-6; +5.0pp over random triggering, p=1.0e-4; paired-trajectory bootstrap CIs exclude zero. AEGIS activates the stronger policy on only 38% of steps, so the lever is timing rather than compute. The probe clears its precondition with an early-window AUROC of 0.764, 95% CI [0.70, 0.84], read from the weak-policy path over the first 30% of trajectory steps before any handoff. We pre-register the full analysis plan, including a conditional recovered-task-rate estimand and explicit kill criteria, and confirm the result on 700 common-random-number episodes per arm, with nA-fail=646.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 4 Jun 2026.
Builds on
20 of 29 references.
References · 20
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
- 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
- 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
- 2025SAFE: Multitask Failure Detection for Vision-Language-Action Modelscited by 80
- 2025RynnVLA-002: A Unified Vision-Language-Action and World Modelcited by 74
- 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 69
- 2025Failure Prediction at Runtime for Generative Robot Policiescited by 42
- 2025FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Modelscited by 35
- 2025FPC-VLA: A Vision-Language-Action Framework with a Supervisor for Failure Prediction and Correctioncited by 21
- 2026Accurate Failure Prediction in Agents Does Not Imply Effective Failure Preventioncited by 15
- 2026LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policiescited by 14
8 more
- 2026Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulationcited by 11
- 2026HELM: Harness-Enhanced Long-horizon Memory for Vision-Language-Action Manipulationcited by 6
- 2026ReconVLA: An Uncertainty-Guided and Failure-Aware Vision-Language-Action Framework for Robotic Controlcited by 5
- 2026Pre-VLA: Preemptive Runtime Verification for Reliable Vision-Language-Action and World-Model Rolloutscited by 5
- 2026Memory-Bound but Not Bandwidth-Limited: The Physical AI Inference Gap in Batch-1 LLM Decodecited by 4
- 2025INSIGHT: INference-time Sequence Introspection for Generating Help Triggers in Vision-Language-Action Modelscited by 3
- 2026AURA: Action-Gated Memory for Robot Policies at Constant VRAMcited by 2
- 2026Confidence-Gated Robot Autonomy: When Does Uncertainty Actually Help?cited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Memory as a Wasting Asset: Pricing Flash Endurance for Embodied Agents, and the Limits of Doing Socited by 0
- 2026Think Fast, Plan Selectively: Adaptive Deliberation for Efficient Data-Driven MPCcited by 0
- 2026RESCUE: Is This Alarm Worth Acting On? Counterfactual Evaluation of VLA Runtime Monitorscited by 0
- 2026RESCUE: Is This Alarm Worth Acting On? Counterfactual Evaluation of VLA Runtime Monitors9 Oct 2026
- 2026Think Fast, Plan Selectively: Adaptive Deliberation for Efficient Data-Driven MPC26 Sep 2026
- 2026Memory as a Wasting Asset: Pricing Flash Endurance for Embodied Agents, and the Limits of Doing So16 Jun 2026