# USER: A Unified and Extensible System for Real-World Online Policy Learning in Embodied AI

arXiv: https://arxiv.org/abs/2602.07837 · HTML: https://mlsanity.com/paper/2602.07837/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2026.

## Built on by (most cited first, 5 of 5)

- [Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies](https://mlsanity.com/paper/2605.00416/) (2026)
- [Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Models](https://mlsanity.com/paper/2602.12628/) (2026)
- [WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning](https://mlsanity.com/paper/2607.29613/) (2026)
- [Dynamic Execution Commitment of Vision-Language-Action Models](https://mlsanity.com/paper/2605.11567/) (2026)
- [Efficient Real-World Online Reinforcement Learning for Robot Manipulation via Centralized Training and Critic Decomposition](https://mlsanity.com/paper/2608.09762/) (2026)

## Abstract (arXiv)

Online policy learning directly in the physical world is a promising yet challenging direction for embodied intelligence. Unlike simulation, real-world systems cannot be arbitrarily accelerated, cheaply reset, or massively replicated, suggesting that real-world policy learning is not merely an algorithmic problem, but inherently a systems problem. We present USER, a underlineUnified and extensible underlineSystunderlineEm for real-world online policy leaunderlineRning. On the systems side, USER introduces a hardware abstraction layer for unified robot management and an adaptive communication plane that enables efficient cloud-edge training. On the learning side, USER adopts a fully asynchronous training framework, designs a persistent and cache-aware replay buffer, and provides extensible abstractions for rewards, algorithms, and policies. Experiments in both simulation and the real world demonstrate that USER supports multi-robot coordination, heterogeneous manipulators, cloud-edge training with large models, and long-running asynchronous training. Together, these capabilities establish USER as a unified and extensible systems foundation for real-world online policy learning.