# Computational Teaching for Driving via Multi-Task Imitation Learning arXiv: https://arxiv.org/abs/2410.01608 ยท HTML: https://mlsanity.com/paper/2410.01608/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. - Builds on 26 of its 58 references that are in the map. - First posted on arXiv 2 Oct 2024. ## Placed in - End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/) - Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/) ## Builds on (most cited first, 26 of 26) - [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019) - [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022) - [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017) - [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017) - [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018) - [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021) - [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020) - [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019) - [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015) - [MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction](https://mlsanity.com/paper/2111.14973/) (2021) - [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020) - [Scene Transformer: A unified architecture for predicting multiple agent trajectories](https://mlsanity.com/paper/2106.08417/) (2021) - [Knowledge Tracing: A Survey](https://mlsanity.com/paper/2201.06953/) (2022) - [An Overview of Machine Teaching](https://mlsanity.com/paper/1801.05927/) (2018) - [Analyzing the Variety Loss in the Context of Probabilistic Trajectory Prediction](https://mlsanity.com/paper/1907.10178/) (2019) - [Multi-Task Imitation Learning for Linear Dynamical Systems](https://mlsanity.com/paper/2212.00186/) (2022) - [Multi-task Hierarchical Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/2305.12633/) (2023) - [Human-oriented Representation Learning for Robotic Manipulation](https://mlsanity.com/paper/2310.03023/) (2023) - [DAISI: Database for AI Surgical Instruction](https://mlsanity.com/paper/2004.02809/) (2020) - [Assistive Teaching of Motor Control Tasks to Humans](https://mlsanity.com/paper/2211.14003/) (2022) - [COACH: Cooperative Robot Teaching](https://mlsanity.com/paper/2302.06199/) (2023) - [A hierarchical adaptive nonlinear model predictive control approach for maximizing tire force usage in autonomous vehicles](https://mlsanity.com/paper/2304.12263/) (2023) - [Generating Language Corrections for Teaching Physical Control Tasks](https://mlsanity.com/paper/2306.07012/) (2023) - [TeachingBot: Robot Teacher for Human Handwriting](https://mlsanity.com/paper/2309.11848/) (2023) - [Uncertainty-aware Self-supervised Learning for Cross-domain Technical Skill Assessment in Robot-assisted Surgery](https://mlsanity.com/paper/2304.14589/) (2023) - [Self-Supervised Instance Segmentation by Grasping](https://mlsanity.com/paper/2305.06305/) (2023) ## Built on by (most cited first, 6 of 6) - [CANINE: Coaching Visually Impaired Users for Interactive Navigation with a Robot Guide Dog](https://mlsanity.com/paper/2605.19501/) (2026) - [On the Strengths and Weaknesses of Data for Open-set Embodied Assistance](https://mlsanity.com/paper/2603.04819/) (2026) - [Fix the Mind, Not the Move: Interpretable AI Assistance via Knowledge-Gap Localization](https://mlsanity.com/paper/2606.05602/) (2026) - [Spatio-Temporal Retrieval-based Priors for Adaptive Computational Teaching in Driving](https://mlsanity.com/paper/2606.25224/) (2026) - [AI Coaching for Accelerating Human Skill Development with Reinforcement Learning](https://mlsanity.com/paper/2606.25337/) (2026) - [Disentangled Skill Representations for Predictive Human Modeling](https://mlsanity.com/paper/2608.23776/) (2026) ## Abstract (arXiv) Learning motor skills for sports or performance driving is often done with professional instruction from expert human teachers, whose availability is limited. Our goal is to enable automated teaching via a learned model that interacts with the student similar to a human teacher. However, training such automated teaching systems is limited by the availability of high-quality annotated datasets of expert teacher and student interactions that are difficult to collect at scale. To address this data scarcity problem, we propose an approach for training a coaching system for complex motor tasks such as high performance driving via a Multi-Task Imitation Learning (MTIL) paradigm. MTIL allows our model to learn robust representations by utilizing self-supervised training signals from more readily available non-interactive datasets of humans performing the task of interest. We validate our approach with (1) a semi-synthetic dataset created from real human driving trajectories, (2) a professional track driving instruction dataset, (3) a track-racing driving simulator human-subject study, and (4) a system demonstration on an instrumented car at a race track. Our experiments show that the right set of auxiliary machine learning tasks improves performance in predicting teaching instructions. Moreover, in the human subjects study, students exposed to the instructions from our teaching system improve their ability to stay within track limits, and show favorable perception of the model's interaction with them, in terms of usefulness and satisfaction.