# GABRIL: Gaze-Based Regularization for Mitigating Causal Confusion in Imitation Learning

arXiv: https://arxiv.org/abs/2507.19647 · HTML: https://mlsanity.com/paper/2507.19647/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 25 Jul 2025.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 20 of 20)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving](https://mlsanity.com/paper/2406.03877/) (2024)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Fighting Copycat Agents in Behavioral Cloning from Observation Histories](https://mlsanity.com/paper/2010.14876/) (2020)
- [AGIL: Learning Attention from Human for Visuomotor Tasks](https://mlsanity.com/paper/1806.03960/) (2018)
- [Causally Regularized Learning with Agnostic Data Selection Bias](https://mlsanity.com/paper/1708.06656/) (2017)
- [Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlation](https://mlsanity.com/paper/2307.07907/) (2023)
- [Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning](https://mlsanity.com/paper/2110.14118/) (2021)
- [Resolving Copycat Problems in Visual Imitation Learning via Residual Action Prediction](https://mlsanity.com/paper/2207.09705/) (2022)
- [ViSaRL: Visual Reinforcement Learning Guided by Human Saliency](https://mlsanity.com/paper/2403.10940/) (2024)
- [Gaze Training by Modulated Dropout Improves Imitation Learning](https://mlsanity.com/paper/1904.08377/) (2019)
- [Memory-based gaze prediction in deep imitation learning for robot manipulation](https://mlsanity.com/paper/2202.04877/) (2022)
- [Selective Eye-gaze Augmentation To Enhance Imitation Learning In Atari Games](https://mlsanity.com/paper/2012.03145/) (2020)

## Built on by (most cited first, 3 of 3)

- [AutoFocus-IL: VLM-based Saliency Maps for Data-Efficient Visual Imitation Learning without Extra Human Annotations](https://mlsanity.com/paper/2511.18617/) (2025)
- [Workspace Models: Lightweight Robotic Memory via Saliency-Driven Supervision](https://mlsanity.com/paper/2609.20820/) (2026)
- [Neurosymbolic Imitation Learning with Human Guidance: A Privileged Information Approach](https://mlsanity.com/paper/2605.07166/) (2026)

## Abstract (arXiv)

Imitation Learning (IL) is a widely adopted approach which enables agents to learn from human expert demonstrations by framing the task as a supervised learning problem. However, IL often suffers from causal confusion, where agents misinterpret spurious correlations as causal relationships, leading to poor performance in testing environments with distribution shift. To address this issue, we introduce GAze-Based Regularization in Imitation Learning (GABRIL), a novel method that leverages the human gaze data gathered during the data collection phase to guide the representation learning in IL. GABRIL utilizes a regularization loss which encourages the model to focus on causally relevant features identified through expert gaze and consequently mitigates the effects of confounding variables. We validate our approach in Atari environments and the Bench2Drive benchmark in CARLA by collecting human gaze datasets and applying our method in both domains. Experimental results show that the improvement of GABRIL over behavior cloning is around 179% more than the same number for other baselines in the Atari and 76% in the CARLA setup. Finally, we show that our method provides extra explainability when compared to regular IL agents.