mlsanityData to 14 Oct 2026

Autonomous Human-Robot Interaction via Operator Imitation

arXiv 2504.02724 ↗First posted 3 Apr 2025cs.RO cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Teleoperated robotic characters can perform expressive interactions with humans, relying on the operators' experience and social intuition. In this work, we propose to create autonomous interactive robots, by training a model to imitate operator data. Our model is trained on a dataset of human-robot interactions, where an expert operator is asked to vary the interactions and mood of the robot, while the operator commands as well as the pose of the human and robot are recorded. Our approach learns to predict continuous operator commands through a diffusion process and discrete commands through a classifier, all unified within a single transformer architecture. We evaluate the resulting model in simulation and with a user study on the real system. We show that our method enables simple autonomous human-robot interactions that are comparable to the expert-operator baseline, and that users can recognize the different robot moods as generated by our model. Finally, we demonstrate a zero-shot transfer of our model onto a different robotic platform with the same operator interface.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 35 of its 50 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

35 of 50 references.

References · 35

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  5. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  6. 2022Video Diffusion Modelscited by 2,305
  7. 2022Human Motion Diffusion Modelcited by 1,080
  8. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 218
  9. 2024Expressive Whole-Body Control for Humanoid Robotscited by 139
  10. 2024MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpaintingcited by 116
  11. 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 109
  12. 2020Learning Latent Representations to Influence Multi-Agent Interactioncited by 93
23 more
  1. 2024HOVER: Versatile Neural Whole-Body Controller for Humanoid Robotscited by 91
  2. 2024Taming Diffusion Probabilistic Models for Character Controlcited by 57
  3. 2023Physically Plausible Full-Body Hand-Object Interaction Synthesiscited by 52
  4. 2014Efficient Model Learning for Human-Robot Collaborative Taskscited by 35
  5. 2023Learning Human-to-Robot Handovers from Point Cloudscited by 34
  6. 2023To the Noise and Back: Diffusion for Shared Autonomycited by 22
  7. 2025Design and Control of a Bipedal Robotic Charactercited by 18
  8. 2021Co-GAIL: Learning Diverse Strategies for Human-Robot Collaborationcited by 17
  9. 2024Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimickingcited by 16
  10. 2023Diffusion Co-Policy for Synergistic Human-Robot Collaborative Taskscited by 15
  11. 2023SynH2R: Synthesizing Hand-Object Motions for Learning Human-to-Robot Handoverscited by 13
  12. 2018Intrinsically motivated reinforcement learning for human-robot interaction in the real-worldcited by 12
  13. 2022It Takes Two: Learning to Plan for Human-Robot Cooperative Carryingcited by 12
  14. 2019Imitating by generating: deep generative models for imitation of interactive taskscited by 11
  15. 2024APRICOT: Active Preference Learning and Constraint-Aware Task Planning with LLMscited by 11
  16. 2023Human-Robot Gym: Benchmarking Reinforcement Learning in Human-Robot Collaborationcited by 9
  17. 2017Robot gains Social Intelligence through Multimodal Deep Reinforcement Learningcited by 7
  18. 2024Robot Interaction Behavior Generation based on Social Motion Forecasting for Human-Robot Interactioncited by 7
  19. 2022MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interactioncited by 6
  20. 2021The Six Hug Commandments: Design and Evaluation of a Human-Sized Hugging Robot with Visual and Haptic Perceptioncited by 5
  21. 2019Demonstration-Guided Deep Reinforcement Learning of Control Policies for Dexterous Human-Robot Interactioncited by 3
  22. 2021Human-Robot Handshaking: A Reviewcited by 3
  23. 2020A Review of Personality in Human Robot Interactionscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Robot Crash Course: Learning Soft and Stylized Fallingcited by 4
  2. 2025Structured Imitation Learning of Interactive Policies through Inverse Gamescited by 0
  3. 2026MulDP: Multimodal Diffusion Policy for Autonomous Quadruped Parkour Navigation across Complex Terrainscited by 0