# Intuitive Programming, Adaptive Task Planning, and Dynamic Role Allocation in Human-Robot Collaboration

arXiv: https://arxiv.org/abs/2511.08732 · HTML: https://mlsanity.com/paper/2511.08732/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 151 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Builds on (most cited first, 30 of 48)

- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Language to Rewards for Robotic Skill Synthesis](https://mlsanity.com/paper/2306.08647/) (2023)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2108.05877/) (2021)
- [RoboDreamer: Learning Compositional World Models for Robot Imagination](https://mlsanity.com/paper/2404.12377/) (2024)
- [HumanPlus: Humanoid Shadowing and Imitation from Humans](https://mlsanity.com/paper/2406.10454/) (2024)
- [Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation](https://mlsanity.com/paper/1707.03374/) (2017)
- [The Surprising Effectiveness of Representation Learning for Visual Imitation](https://mlsanity.com/paper/2112.01511/) (2021)
- [A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms](https://mlsanity.com/paper/1907.03146/) (2019)
- [Yell At Your Robot: Improving On-the-Fly from Language Corrections](https://mlsanity.com/paper/2403.12910/) (2024)
- [Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos](https://mlsanity.com/paper/2103.16817/) (2021)
- [AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos](https://mlsanity.com/paper/1912.04443/) (2019)
- [Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1907.13098/) (2019)
- [GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstration](https://mlsanity.com/paper/2311.12015/) (2023)
- [Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller](https://mlsanity.com/paper/1911.09676/) (2019)
- [Learning Generalizable Manipulation Policies with Object-Centric 3D Representations](https://mlsanity.com/paper/2310.14386/) (2023)
- [Graph-Structured Visual Imitation](https://mlsanity.com/paper/1907.05518/) (2019)
- [Demo2Code: From Summarizing Demonstrations to Synthesizing Code via Extended Chain-of-Thought](https://mlsanity.com/paper/2305.16744/) (2023)
- [Trust-Aware Decision Making for Human-Robot Collaboration: Model Learning and Planning](https://mlsanity.com/paper/1801.04099/) (2018)
- [VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation](https://mlsanity.com/paper/2105.09371/) (2021)
- [Thought Cloning: Learning to Think while Acting by Imitating Human Thinking](https://mlsanity.com/paper/2306.00323/) (2023)
- [Learning Object-Action Relations from Bimanual Human Demonstration Using Graph Networks](https://mlsanity.com/paper/1908.08391/) (2019)
- [K-VIL: Keypoints-based Visual Imitation Learning](https://mlsanity.com/paper/2209.03277/) (2022)
- [CoSTAR: Instructing Collaborative Robots with Behavior Trees and Vision](https://mlsanity.com/paper/1611.06145/) (2016)
- [Learning to Reason over Scene Graphs: A Case Study of Finetuning GPT-2 into a Robot Language Model for Grounded Task Planning](https://mlsanity.com/paper/2305.07716/) (2023)
- [A Hierarchical Architecture for Human-Robot Cooperation Processes](https://mlsanity.com/paper/2009.02807/) (2020)
- [ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Space](https://mlsanity.com/paper/2309.05310/) (2023)
- [Optimal task and motion planning and execution for human-robot multi-agent systems in dynamic environments](https://mlsanity.com/paper/2303.14874/) (2023)

## Built on by (most cited first, 1 of 1)

- [Human-Aware Robot Behaviour in Self-Driving Labs](https://mlsanity.com/paper/2603.08420/) (2026)

## Abstract (arXiv)

Remarkable capabilities have been achieved by robotics and AI, mastering complex tasks and environments. Yet, humans often remain passive observers, fascinated but uncertain how to engage. Robots, in turn, cannot reach their full potential in human-populated environments without effectively modeling human states and intentions and adapting their behavior. To achieve a synergistic human-robot collaboration (HRC), a continuous information flow should be established: humans must intuitively communicate instructions, share expertise, and express needs. In parallel, robots must clearly convey their internal state and forthcoming actions to keep users informed, comfortable, and in control. This review identifies and connects key components enabling intuitive information exchange and skill transfer between humans and robots. We examine the full interaction pipeline: from the human-to-robot communication bridge translating multimodal inputs into robot-understandable representations, through adaptive planning and role allocation, to the control layer and feedback mechanisms to close the loop. Finally, we highlight trends and promising directions toward more adaptive, accessible HRC.