mlsanityData to 14 Oct 2026

Learning Temporal Strategic Relationships using Generative Adversarial Imitation Learning

arXiv 1805.04969 ↗First posted 13 May 2018cs.CVcited by 3

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

This paper presents a novel framework for automatic learning of complex strategies in human decision making. The task that we are interested in is to better facilitate long term planning for complex, multi-step events. We observe temporal relationships at the subtask level of expert demonstrations, and determine the different strategies employed in order to successfully complete a task. To capture the relationship between the subtasks and the overall goal, we utilise two external memory modules, one for capturing dependencies within a single expert demonstration, such as the sequential relationship among different sub tasks, and a global memory module for modelling task level characteristics such as best practice employed by different humans based on their domain expertise. Furthermore, we demonstrate how the hidden state representation of the memory can be used as a reward signal to smooth the state transitions, eradicating subtle changes. We evaluate the effectiveness of the proposed model for an autonomous highway driving application, where we demonstrate its capability to learn different expert policies and outperform state-of-the-art methods. The scope in industrial applications extends to any robotics and automation application which requires learning from complex demonstrations containing series of subtasks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 22 of its 48 references that are in the map. First posted on arXiv 13 May 2018.

Builds on

22 of 48 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  3. 2016Generative Adversarial Imitation Learningcited by 1,597
  4. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  5. 2015Prioritized Experience Replaycited by 1,099
  6. 2017Wasserstein GANcited by 639
  7. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  8. 2017One-Shot Imitation Learningcited by 233
  9. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 231
  10. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
  11. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168
  12. 2015Ask Me Anything: Dynamic Memory Networks for Natural Language Processingcited by 117
10 more
  1. 2017Third-Person Imitation Learningcited by 103
  2. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 93
  3. 2017Neural Map: Structured Memory for Deep Reinforcement Learningcited by 90
  4. 2017Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detectioncited by 42
  5. 2014Real time Detection of Lane Markers in Urban Streetscited by 41
  6. 2017Simultaneous Policy Learning and Latent State Inference for Imitating Driver Behaviorcited by 16
  7. 2017InfoGAIL: Interpretable Imitation Learning from Visual Demonstrationscited by 11
  8. 2018Tracking by Prediction: A Deep Generative Model for Mutli-Person localisation and Trackingcited by 9
  9. 2017Tree Memory Networks for Modelling Long-term Temporal Dependenciescited by 7
  10. 2018Task Specific Visual Saliency Prediction with Memory Augmented Conditional Generative Adversarial Networkscited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 0; 2023: 0; 2024: 2.

3 papers.

  1. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 58
  2. 2024World Models with Hints of Large Language Models for Goal Achievingcited by 6
  3. 2024BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplaycited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2024Interpretable Imitation Learning via Generative Adversarial STL Inference and Control
  2. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving
  3. 2018Deep Imitative Models for Flexible Inference, Planning, and Control
  4. 2020State-only Imitation with Transition Dynamics Mismatch
  5. 2018Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information
  6. 2019A New Framework for Query Efficient Active Imitation Learning
  7. 2021Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning
  8. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations