mlsanityData to 14 Oct 2026

Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperation

arXiv 2309.01952 ↗First posted 5 Sep 2023cs.ROcited by 54

On the map: Offline RL and imitation learning from data → Real-world robot offline RL

Abstract (arXiv)

We tackle the problem of developing humanoid loco-manipulation skills with deep imitation learning. The difficulty of collecting task demonstrations and training policies for humanoids with a high degree of freedom presents substantial challenges. We introduce TRILL, a data-efficient framework for training humanoid loco-manipulation policies from human demonstrations. In this framework, we collect human demonstration data through an intuitive Virtual Reality (VR) interface. We employ the whole-body control formulation to transform task-space commands by human operators into the robot's joint-torque actuation while stabilizing its dynamics. By employing high-level action abstractions tailored for humanoid loco-manipulation, our method can efficiently learn complex sensorimotor skills. We demonstrate the effectiveness of TRILL in simulation and on a real-world robot for performing various loco-manipulation tasks. Videos and additional materials can be found on the project page: https://ut-austin-rpl.github.io/TRILL.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 45 references that are in the map. First posted on arXiv 5 Sep 2023.

Builds on

19 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulation6 Aug

Other references · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  3. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  4. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 289
  5. 2020Critic Regularized Regressioncited by 275
  6. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 153
  7. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 136
  8. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  9. 2021Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulationcited by 59
  10. 2020Deep Imitation Learning for Bimanual Robotic Manipulationcited by 50
  11. 2020ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulationcited by 49
  12. 2023Teleoperation of Humanoid Robots: A Surveycited by 34
6 more
  1. 2020Learning Multi-Arm Manipulation Through Collaborative Teleoperationcited by 32
  2. 2023Dynamic Mobile Manipulation via Whole-Body Bilateral Teleoperation of a Wheeled Humanoidcited by 7
  3. 2019Deploying the NASA Valkyrie Humanoid for IED Response: An Initial Approach and Evaluation Summarycited by 3
  4. 2022A Fast, Autonomous, Bipedal Walking Behavior over Rapid Regionscited by 3
  5. 2019Finding Locomanipulation Plans Quickly in the Locomotion Constrained Manifoldcited by 1
  6. 2022Control and Evaluation of a Humanoid Robot with Rolling Contact Kneescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 25; 2025: 21; 2026: 8.

54 papers.

Sort:

By area:

  1. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  2. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 189
  3. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 185
  4. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
  5. 2024Expressive Whole-Body Control for Humanoid Robotscited by 139
  6. 2024Flow as the Cross-Domain Manipulation Interfacecited by 132
  7. 2024Learning Human-to-Humanoid Real-Time Whole-Body Teleoperationcited by 130
  8. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 116
  9. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 116
  10. 2024ExBody2: Advanced Expressive Humanoid Whole-Body Controlcited by 88
  11. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 76
  12. 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interactioncited by 71
38 more
  1. 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 67
  2. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 56
  3. 2024Generalizable Humanoid Manipulation with 3D Diffusion Policiescited by 52
  4. 2024Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Controlcited by 51
  5. 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 51
  6. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 46
  7. 2024CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamicscited by 44
  8. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 31
  9. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 30
  10. 2024Learning from Massive Human Videos for Universal Humanoid Pose Controlcited by 29
  11. 2025Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulationcited by 25
  12. 2024Mamba Policy: Towards Efficient 3D Diffusion Policy with Hybrid Selective State Modelscited by 23
  13. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 21
  14. 2024Strategy and Skill Learning for Physics-based Table Tennis Animationcited by 20
  15. 2024Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimickingcited by 16
  16. 2025Tool-as-Interface: Learning Robot Policies from Observing Human Tool Usecited by 8
  17. 2025XRoboToolkit: A Cross-Platform Framework for Robot Teleoperationcited by 8
  18. 2025Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactionscited by 8
  19. 2024Deep Reinforcement Learning for Bipedal Locomotion: A Brief Surveycited by 7
  20. 2024Whole-Body Teleoperation for Mobile Manipulation at Zero Added Costcited by 6
  21. 2024XBG: End-to-end Imitation Learning for Autonomous Behaviour in Human-Robot Interaction and Collaborationcited by 4
  22. 2025Latent Adaptive Planner for Dynamic Manipulationcited by 4
  23. 2025Gaze-Guided Task Decomposition for Imitation Learning in Robotic Manipulationcited by 3
  24. 2025Sample from What You See: Visuomotor Policy Learning via Diffusion Bridge with Observation-Embedded Stochastic Differential Equationcited by 3
  25. 2024The Duke Humanoid: Design and Control For Energy Efficient Bipedal Locomotion Using Passive Dynamicscited by 2
  26. 2025TeleOpBench: A Simulator-Centric Benchmark for Dual-Arm Dexterous Teleoperationcited by 2
  27. 2025A Unified and Fast-Sampling Diffusion Bridge Framework via Stochastic Optimal Controlcited by 2
  28. 2026Humanoid-DART: Humanoid Loco-Manipulation using Diffusion-guided Augmentation through Relabeling and Trackingcited by 2
  29. 2025Improving Low-Cost Teleoperation: Augmenting GELLO with Forcecited by 1
  30. 2026AdaptManip: Learning Adaptive Whole-Body Object Lifting and Delivery with Online Recurrent State Estimationcited by 1
  31. 2026DreamControl-v2: Simpler and Scalable Autonomous Humanoid Skills via Trainable Guided Diffusion Priorscited by 1
  32. 2026InDex: Empowering VLA Models with Intent-Conditioned Arm-Hand Coordination for Dexterous Manipulationcited by 1
  33. 2024AsymDex: Asymmetry and Relative Coordinates for RL-based Bimanual Dexteritycited by 0
  34. 2025Physically Consistent Humanoid Loco-Manipulation using Latent Diffusion Modelscited by 0
  35. 2025Coupled Distributional Random Expert Distillation for World Model Online Imitation Learningcited by 0
  36. 2025Imitation Learning for Active Neck Motion Enabling Robot Manipulation beyond the Field of Viewcited by 0
  37. 2025Learning Pivoting Manipulation with Force and Vision Feedback Using Optimization-based Demonstrationscited by 0
  38. 2025Kinematics-Aware Multi-Policy Reinforcement Learning for Force-Capable Humanoid Loco-Manipulationcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning
  2. 2025WholeBodyVLA: Towards Unified Latent VLA for Whole-Body Loco-Manipulation Control
  3. 2025Humanoid Locomotion and Manipulation: Current Progress and Challenges in Control, Planning, and Learning
  4. 2025SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blending
  5. 2026SIMPLE: Simulation-Based Policy Learning and Evaluation for Humanoid Loco-manipulation
  6. 2025DemoHLM: From One Demonstration to Generalizable Humanoid Loco-Manipulation
  7. 2025Learning Adaptive Neural Teleoperation for Humanoid Robots: From Inverse Kinematics to End-to-End Control ↗
  8. 2026LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World