mlsanityData to 7 Oct 2026

BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplay

arXiv 2402.14194 ↗First posted 22 Feb 2024cs.LG cs.ROcited by 3

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning learns a policy from demonstrations without requiring hand-designed reward functions. In many robotic tasks, such as autonomous racing, imitated policies must model complex environment dynamics and human decision-making. Sequence modeling is highly effective in capturing intricate patterns of motion sequences but struggles to adapt to new environments or distribution shifts that are common in real-world robotics tasks. In contrast, Adversarial Imitation Learning (AIL) can mitigate this effect, but struggles with sample inefficiency and handling complex motion patterns. Thus, we propose BeTAIL: Behavior Transformer Adversarial Imitation Learning, which combines a Behavior Transformer (BeT) policy from human demonstrations with online AIL. BeTAIL adds an AIL residual policy to the BeT policy to model the sequential decision-making process of human experts and correct for out-of-distribution states or shifts in environment dynamics. We test BeTAIL on three challenges with expert-level demonstrations of real human gameplay in Gran Turismo Sport. Our proposed residual BeTAIL reduces environment interactions and improves racing performance and stability, even when the BeT is pretrained on different tracks than downstream learning. Videos and code available at: https://sites.google.com/berkeley.edu/BeTAIL/home.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 42 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

29 of 42 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges5 Sep
  2. 2023How To Guide Your Learner: Imitation Learning with Active Adaptive Expert Involvement3 Mar
  3. 2021What Matters for Adversarial Imitation Learning?1 Jun
  4. 2018Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information29 Sep
  5. 2018Learning Temporal Strategic Relationships using Generative Adversarial Imitation Learning13 May

Other references · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  6. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  7. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  8. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  9. 2022Online Decision Transformercited by 194
  10. 2018Residual Policy Learningcited by 146
  11. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 92
  12. 2020Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learningcited by 60
12 more
  1. 2022Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racingcited by 56
  2. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  3. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 43
  4. 2021Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learningcited by 41
  5. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioningcited by 24
  6. 2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 23
  7. 2023Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstrationcited by 14
  8. 2023Guarded Policy Optimization with Imperfect Online Demonstrationscited by 10
  9. 2020A Taxonomy and Review of Algorithms for Modeling and Predicting Human Driver Behaviorcited by 9
  10. 2022A Gaussian Process Model for Opponent Prediction in Autonomous Racingcited by 5
  11. 2023Residual Policy Learning for Vehicle Control of Autonomous Racing Carscited by 4
  12. 2022Task Phasing: Automated Curriculum Learning from Demonstrationscited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2024Dreaming to Assist: Learning to Align with Human Objectives for Shared Control in High-Speed Racingcited by 7
  2. 2025Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulationcited by 1
  3. 2025Structured Imitation Learning of Interactive Policies through Inverse Gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020ADAIL: Adaptive Adversarial Imitation Learning
  2. 2026Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learning
  3. 2023DiffAIL: Diffusion Adversarial Imitation Learning
  4. 2025SD2AIL: Adversarial Imitation Learning from Synthetic Demonstrations via Diffusion Models
  5. 2023Generative Adversarial Neuroevolution for Control Behaviour Imitation
  6. 2021RAIL: A modular framework for Reinforcement-learning-based Adversarial Imitation Learning
  7. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards
  8. 2023Policy Contrastive Imitation Learning