mlsanityData to 7 Oct 2026

Student-Informed Teacher Training

arXiv 2412.09149 ↗First posted 12 Dec 2024cs.RO cs.LGcited by 6

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning with a privileged teacher has proven effective for learning complex control behaviors from high-dimensional inputs, such as images. In this framework, a teacher is trained with privileged task information, while a student tries to predict the actions of the teacher with more limited observations, e.g., in a robot navigation task, the teacher might have access to distances to nearby obstacles, while the student only receives visual observations of the scene. However, privileged imitation learning faces a key challenge: the student might be unable to imitate the teacher's behavior due to partial observability. This problem arises because the teacher is trained without considering if the student is capable of imitating the learned behavior. To address this teacher-student asymmetry, we propose a framework for joint training of the teacher and student policies, encouraging the teacher to learn behaviors that can be imitated by the student despite the latters' limited access to information and its partial observability. Based on the performance bound in imitation learning, we add (i) the approximated action difference between teacher and student as a penalty term to the reward function of the teacher, and (ii) a supervised teacher-student alignment step. We motivate our method with a maze navigation task and demonstrate its effectiveness on complex vision-based quadrotor flight and manipulation tasks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 39 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

26 of 39 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Sequence Model Imitation Learning with Unobserved Contexts3 Aug
  2. 2020Bridging the Imitation Gap by Adaptive Insubordination23 Jul
  3. 2018An Algorithmic Perspective on Imitation Learning16 Nov

Other references · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  6. 2019Learning by Cheatingcited by 361
  7. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 202
  8. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 132
  9. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 116
  10. 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
  11. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 98
  12. 2020Flightmare: A Flexible Quadrotor Simulatorcited by 69
11 more
  1. 2023Multi-Stage Cable Routing through Hierarchical Imitation Learningcited by 59
  2. 2020Can Increasing Input Dimensionality Improve Deep Reinforcement Learning?cited by 37
  3. 2020Robust Asymmetric Learning in POMDPscited by 23
  4. 2023On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfercited by 20
  5. 2024Privileged Sensing Scaffolds Reinforcement Learningcited by 18
  6. 2022Leveraging Fully Observable Policies for Learning under Partial Observabilitycited by 17
  7. 2023TGRL: An Algorithm for Teacher Guided Reinforcement Learningcited by 16
  8. 2015Data Generation as Sequential Decision Makingcited by 15
  9. 2024Designing Skill-Compatible AI: Methodologies and Frameworks in Chesscited by 10
  10. 2024SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experiencecited by 10
  11. 2023Bridging the Sim-to-Real Gap from the Information Bottleneck Perspectivecited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Modelscited by 10
  2. 2025To Distill or Decide? Understanding the Algorithmic Trade-off in Partially Observable Reinforcement Learningcited by 3
  3. 2026Generative Event Pretraining with Foundation Model Alignmentcited by 3
  4. 2025Distilling Realizable Students from Unrealizable Teacherscited by 2
  5. 2026Approximate Imitation Learning for Event-based Quadrotor Flight in Cluttered Environmentscited by 1
  6. 2026Teacher-Student Representational Alignment for Reinforcement Learning-Driven Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Active Imitation Learning from Multiple Non-Deterministic Teachers: Formulation, Challenges, and Algorithms
  2. 2026A Good Self-Teacher Meets the Student Where They Are: Joint On-Policy Learning and Teaching
  3. 2024A Survey of Imitation Learning Methods, Environments and Metrics
  4. 2026Neurosymbolic Imitation Learning with Human Guidance: A Privileged Information Approach
  5. 2023Curriculum-Based Imitation of Versatile Skills
  6. 2023Information Maximizing Curriculum: A Curriculum-Based Approach for Imitating Diverse Skills
  7. 2022Causal Imitation Learning with Unobserved Confounders
  8. 2021Iterative Teacher-Aware Learning