Resilience Matters for Embodied Agents System: New Metrics, Systematic Evaluation, and Optimization
Not on the map. Why?
Abstract (arXiv)
Embodied Agents System (EAS) are increasingly deployed in open-world physical domains, where reliability directly dictates deployment quality and human-agent trust. However, existing evaluations rely on outcome-centric metrics as success rate or safety scores that collapse diverse execution trajectories into coarse scores, obscuring the dynamic processes underlying agent behavior. Therefore, they ignore a critical property of EAS -- which we define as the Resilience -- that reflects how EASs recover, stabilize, and extend under perturbations and across iterative updates. The lack of resilience is particularly critical in open-world environments due to continuous unexpected disruptions, thus directly affecting the quality of EAS deployment. To address this problem, we gain insight from the resilience-engineering concepts to EAS groundings and propose a novel resilience evaluation framework that can be flexibly applied to any EAS. Specifically, we define the first comprehensive resilience metrics suite for EASs system that exposes Rebound, Stability, and Graceful Extensibility across embodied tasks execution, providing a practical grounding for EAS resilience analysis. We further implement the resilience evaluation layer that transforms execution process into assessments for diagnosis and optimization. Across 400 household tasks with 10 EAS, we reveal the process-level distinction hidden by outcome metrics, including recovery cost differences among successful episodes (Delta Crec=25.2), increased instability and task-family degradation. Metrics-guided optimizations reduce recovery cost and increase stability, graceful extensibility completion, showing the diagnostic effect of resilience evaluation. Our results reveal a trade-off among resilience characteristics, suggesting that a resilient EAS construction should be configured according to deployment-specific requirements.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 24 Aug 2026.
Builds on
29 of 41 references.
References · 29
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
- 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
- 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
- 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Makingcited by 142
17 more
- 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 102
- 2025AgentEvolver: Towards Efficient Self-Evolving Agent Systemcited by 77
- 2023DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignmentcited by 58
- 2025Embodied AI Agents: Modeling the Worldcited by 57
- 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 40
- 2023CoPAL: Corrective Planning of Robot Actions with Large Language Modelscited by 39
- 2024RePLan: Robotic Replanning with Perception and Language Modelscited by 39
- 2024EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agentscited by 24
- 2021Success Weighted by Completion Time: A Dynamics-Aware Evaluation Criteria for Embodied Navigationcited by 21
- 2024The Current Challenges of Software Engineering in the Era of Large Language Modelscited by 21
- 2026CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decodingcited by 13
- 2021Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systemscited by 12
- 2026Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of Large Language Model Agentscited by 11
- 2025Embodied Arena: A Comprehensive, Unified, and Evolving Evaluation Platform for Embodied AIcited by 7
- 2026HazardArena: Evaluating Semantic Safety in Vision-Language-Action Modelscited by 6
- 2026Using large language models for embodied planning introduces systematic safety riskscited by 4
- 2025MTTR-A: Measuring Cognitive Recovery Latency in Multi-Agent Systemscited by 2
Built on by
Not cited yet.