mlsanityData to 14 Oct 2026

UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers

arXiv 2407.10353 ↗First posted 14 Jul 2024cs.ROcited by 75

Not on the map. Why?

Abstract (arXiv)

We introduce UMI-on-Legs, a new framework that combines real-world and simulation data for quadruped manipulation systems. We scale task-centric data collection in the real world using a hand-held gripper (UMI), providing a cheap way to demonstrate task-relevant manipulation skills without a robot. Simultaneously, we scale robot-centric data in simulation by training whole-body controller for task-tracking without task simulation setups. The interface between these two policies is end-effector trajectories in the task frame, inferred by the manipulation policy and passed to the whole-body controller for tracking. We evaluate UMI-on-Legs on prehensile, non-prehensile, and dynamic manipulation tasks, and report over 70% success rate on all tasks. Lastly, we demonstrate the zero-shot cross-embodiment deployment of a pre-trained manipulation policy checkpoint from prior work, originally intended for a fixed-base robot arm, on our quadruped system. We believe this framework provides a scalable path towards learning expressive manipulation skills on dynamic robot embodiments. Please checkout our website for robot videos, code, and data: https://umi-on-legs.github.io

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Jul 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 37; 2026: 27.

75 papers.

Sort:

By area:

  1. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 195
  2. 2024ExBody2: Advanced Expressive Humanoid Whole-Body Controlcited by 88
  3. 2025Humanoid Policy ~ Human Policycited by 78
  4. 2025HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpitcited by 76
  5. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 75
  6. 2026Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalizationcited by 66
  7. 2025RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learningcited by 54
  8. 2024Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Controlcited by 51
  9. 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 51
  10. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  11. 2025DexWild: Dexterous Human Interactions for In-the-Wild Robot Policiescited by 38
  12. 2024ForceMimic: Force-Centric Imitation Learning with Force-Motion Capture System for Contact-Rich Manipulationcited by 36
38 more
  1. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 31
  2. 2025Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Grippercited by 29
  3. 2024TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learningcited by 22
  4. 2025SKIL: Semantic Keypoint Imitation Learning for Generalizable Data-efficient Manipulationcited by 21
  5. 2025Learning coordinated badminton skills for legged manipulatorscited by 19
  6. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 16
  7. 2025Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfercited by 15
  8. 2024Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushingcited by 13
  9. 2025Bridging the Sim-to-Real Gap for Athletic Loco-Manipulationcited by 13
  10. 2026Humanoid Manipulation Interface: Humanoid Whole-Body Manipulation from Robot-Free Demonstrationscited by 12
  11. 2025FACET: Force-Adaptive Control via Impedance Reference Tracking for Legged Robotscited by 11
  12. 2025UMI-on-Air: Embodiment-Aware Guidance for Embodiment-Agnostic Visuomotor Policiescited by 11
  13. 2025Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretrainingcited by 9
  14. 2025ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for Long-Horizon Taskscited by 9
  15. 2025MV-UMI: A Scalable Multi-View Interface for Cross-Embodiment Learningcited by 9
  16. 2024Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Modelscited by 8
  17. 2025CodeDiffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguitycited by 8
  18. 2025Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoidscited by 7
  19. 2025VAMOS: A Hierarchical Vision-Language-Action Model for Capability-Modulated and Steerable Navigationcited by 7
  20. 2026π, But Make It Fly: Physics-Guided Transfer of VLA Models to Aerial Manipulationcited by 7
  21. 2024Whole-body End-Effector Pose Trackingcited by 6
  22. 2024WildLMa: Long Horizon Loco-Manipulation in the Wildcited by 6
  23. 2025R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized Manipulationcited by 6
  24. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 6
  25. 2025Dynamic object goal pushing with mobile manipulators through model-free constrained reinforcement learningcited by 5
  26. 2025A Practical Guide for Incorporating Symmetry in Diffusion Policycited by 5
  27. 20253D Equivariant Visuomotor Policy Learning via Spherical Projectioncited by 5
  28. 2025Versatile Loco-Manipulation through Flexible Interlimb Coordinationcited by 5
  29. 2026HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editingcited by 5
  30. 2025SLIM: Sim-to-Real Legged Instructive Manipulation via Long-Horizon Visuomotor Learningcited by 4
  31. 2025X-Capture: An Open-Source Portable Device for Multi-Sensory Learningcited by 4
  32. 2025DiffusionRL: Efficient Training of Diffusion Policies for Robotic Grasping Using RL-Adapted Large-Scale Datasetscited by 4
  33. 2026UMI-3D: Extending Universal Manipulation Interface from Vision-Limited to 3D Spatial Perceptioncited by 4
  34. 2026HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alonecited by 4
  35. 2024Combining Planning and Diffusion for Mobility with Unknown Dynamicscited by 3
  36. 2025Bipedalism for Quadrupedal Robots: Versatile Loco-Manipulation through Risk-Adaptive Reinforcement Learningcited by 3
  37. 2025Humanoid Whole-Body Badminton via an Annealed Reinforcement Learning Curriculumcited by 3
  38. 2025FALCON: Actively Decoupled Visuomotor Policies for Loco-Manipulation with Foundation-Model-Based Coordinationcited by 3

50 most cited of 75 All 75 (JSON)