mlsanityData to 7 Oct 2026

Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL

arXiv 2608.12063 ↗First posted 12 Aug 2026cs.RO cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Real-world robot offline RL

Abstract (arXiv)

Integrating locomotion and manipulation is essential for robot autonomy, but scaling standard Reinforcement Learning (RL) to complex tasks is severely bottlenecked by the slow, manual process of dense reward shaping. To bypass this limitation, we leverage Sample-based Model Predictive Control (SMPC) entirely in simulation as an automated, rapidly tunable expert to generate massive offline datasets. Because this data solves the fundamental exploration problem, we can train an off-policy RL agent using purely sparse task rewards, drastically reducing the time required to learn new skills and eliminating the need for manual tuning. Integrating this high-level agent with a low-level dynamic stability controller yields more optimal behaviors that strictly align with true task objectives, ultimately allowing the learned policies to surpass the original optimal control teacher. We validate the robustness of this sim-to-real framework by successfully deploying complex loco-manipulation skills across different morphologies, including an arm-equipped Spot quadruped and a G1 humanoid.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 21 of its 24 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

21 of 24 references.

References · 21

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  2. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  3. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  4. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  5. 2018Deep Reinforcement Learning and the Deadly Triadcited by 170
  6. 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 89
  7. 2023Imitation Bootstrapped Reinforcement Learningcited by 54
  8. 2022Predictive Sampling: Real-time Behaviour Synthesis with MuJoCocited by 39
  9. 2025ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learningcited by 34
  10. 2025FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Controlcited by 32
  11. 2023Sim-to-Real Learning for Humanoid Box Loco-Manipulationcited by 30
  12. 2024Opt2Skill: Imitating Dynamically-feasible Whole-Body Trajectories for Versatile Humanoid Loco-Manipulationcited by 28
9 more
  1. 2026mjlab: A Lightweight Framework for GPU-Accelerated Robot Learningcited by 13
  2. 2024Jacta: A Versatile Planner for Learning Dexterous and Whole-body Manipulationcited by 7
  3. 2023Hierarchical Optimization-based Control for Whole-body Loco-manipulation of Heavy Objectscited by 6
  4. 2025Versatile Loco-Manipulation through Flexible Interlimb Coordinationcited by 4
  5. 2022Contact Optimization for Non-Prehensile Loco-Manipulation via Hierarchical Model Predictive Controlcited by 3
  6. 2018Sample-Efficient Learning of Nonprehensile Manipulation Policies via Physics-Based Informed State Distributionscited by 2
  7. 2025Generative Models From and For Sampling-Based MPC: A Bootstrapped Approach For Adaptive Contact-Rich Manipulationcited by 2
  8. 2025A Nonlinear MPC Framework for Loco-Manipulation of Quadrupedal Robots with Non-Negligible Manipulator Dynamicscited by 1
  9. 2025Whole-Body Inverse Dynamics MPC for Legged Loco-Manipulationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Cross-Embodiment Offline Reinforcement Learning for Heterogeneous Robot Datasets
  2. 2023A Real-World Quadrupedal Locomotion Benchmark for Offline Reinforcement Learning
  3. 2026Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
  4. 2021Efficient Robotic Manipulation Through Offline-to-Online Reinforcement Learning and Goal-Aware State Information
  5. 2026Scaling Tasks, Not Samples: Mastering Humanoid Control through Multi-Task Model-Based Reinforcement Learning
  6. 2025GeCCo -- a Generalist Contact-Conditioned Policy for Loco-Manipulation Skills on Legged Robots ↗
  7. 2025DemoHLM: From One Demonstration to Generalizable Humanoid Loco-Manipulation
  8. 2026REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning