A Glimpse into Long-term Physical Coexistence with Intelligent Robots
Not on the map. Why?
Abstract (arXiv)
Long-term physical coexistence with intelligent robots requires more than capable robot policies. A persistent robotic assistant must support diverse user-facing interfaces, maintain long-horizon memory of people and preferences, coordinate across robot embodiments, and translate human intent into safe physical execution. We introduce PHILIA, a multi-robot agent built around a robot gateway abstraction. PHILIA retains the rich interaction and tool ecosystem of OpenClaw while exposing robot-local runtimes, onboard perception, navigation, speaker, and robot policies through a unified capability interface. This design decouples low-frequency, high-semantic agent reasoning from high-frequency, low-level robot execution, enabling plug-and-play integration of user interfaces, robot embodiments, and policy backends. As a result, the user experience becomes compositional: advances in user interfaces, robot embodiments, robot policies, navigation, or interaction algorithms can improve the overall experience without redesigning the system. We validate the architecture on Astribot S1 robots while designing the robot gateway contract to support future heterogeneous robot platforms through a shared capability interface for observation, task execution, navigation, speech playback, status monitoring, and task cancellation. We present representative use cases in which agent memory and scene understanding are grounded in robot actions. These span interactive household scenarios, ranging from simple organization to challenging long-horizon and dexterous service tasks, such as packing a backpack and lifting a garbage bag. We highlight the human-robot interaction flow, where contextual understanding of user intent and preferences, together with human-in-the-loop confirmation or adjustment during execution, is essential for effective assistance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 13 Jul 2026.
Builds on
36 of 52 references.
References · 36
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
24 more
- 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 346
- 2025π*0.6: a VLA That Learns From Experiencecited by 303
- 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 266
- 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 221
- 2025Real-Time Execution of Action Chunking Flow Policiescited by 216
- 2022Robot Operating System 2: Design, Architecture, and Uses In The Wildcited by 199
- 2020LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mappingcited by 188
- 2021FAST-LIO2: Fast Direct LiDAR-inertial Odometrycited by 183
- 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Modelscited by 153
- 2017Behavior Trees in Robotics and AI: An Introductioncited by 107
- 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agentscited by 87
- 2020The Marathon 2: A Navigation Systemcited by 70
- 2025Training-Time Action Conditioning for Efficient Real-Time Chunkingcited by 64
- 2020A Survey of Behavior Trees in Robotics and AIcited by 55
- 2026RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Taskscited by 33
- 2025Large Language Models for Multi-Robot Systems: A Surveycited by 27
- 2025Task adaptation of Vision-Language-Action model: 1st Place Solution for the 2025 BEHAVIOR Challengecited by 15
- 2016The STRANDS Project: Long-Term Autonomy in Everyday Environmentscited by 13
- 2025Towards Human-level Intelligence via Human-like Whole-Body Manipulationcited by 13
- 2026chi0: Resource-Aware Robust Manipulation via Taming Distributional Inconsistenciescited by 12
- 2020Behavior Trees in Action: A Study of Robotics Applicationscited by 9
- 2025Mind to Hand: Purposeful Robotic Control via Embodied Reasoningcited by 8
- 2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planningcited by 7
- 2024Interactive Continual Learning Architecture for Long-Term Personalization of Home Service Robotscited by 6
Built on by
Not cited yet.