# GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation

arXiv: https://arxiv.org/abs/2312.10802 · HTML: https://mlsanity.com/paper/2312.10802/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 17 Dec 2023.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 22 of 22)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning](https://mlsanity.com/paper/1910.11956/) (2019)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research](https://mlsanity.com/paper/1802.09464/) (2018)
- [AlgaeDICE: Policy Gradient from Arbitrary Experience](https://mlsanity.com/paper/1912.02074/) (2019)
- [Imitation Learning via Off-Policy Distribution Matching](https://mlsanity.com/paper/1912.05032/) (2019)
- [Goal-conditioned Imitation Learning](https://mlsanity.com/paper/1906.05838/) (2019)
- [OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation](https://mlsanity.com/paper/2106.10783/) (2021)
- [Imitation Learning from Imperfect Demonstration](https://mlsanity.com/paper/1901.09387/) (2019)
- [Hierarchical Imitation and Reinforcement Learning](https://mlsanity.com/paper/1803.00590/) (2018)
- [Learning and Retrieval from Prior Data for Skill-based Imitation Learning](https://mlsanity.com/paper/2210.11435/) (2022)
- [Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching](https://mlsanity.com/paper/2202.02433/) (2022)
- [Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information](https://mlsanity.com/paper/1810.01266/) (2018)
- [LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimation](https://mlsanity.com/paper/2202.13536/) (2022)
- [Learning Dense Rewards for Contact-Rich Manipulation Tasks](https://mlsanity.com/paper/2011.08458/) (2020)
- [Adversarial Option-Aware Hierarchical Imitation Learning](https://mlsanity.com/paper/2106.05530/) (2021)
- [Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Control](https://mlsanity.com/paper/2210.01969/) (2022)
- [Here's What I've Learned: Asking Questions that Reveal Reward Learning](https://mlsanity.com/paper/2107.01995/) (2021)
- [SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments](https://mlsanity.com/paper/2204.01922/) (2022)

## Built on by (most cited first, 6 of 6)

- [Act2Goal: From World Model To General Goal-conditioned Policy](https://mlsanity.com/paper/2512.23541/) (2025)
- [IDIL: Imitation Learning of Intent-Driven Expert Behavior](https://mlsanity.com/paper/2404.16989/) (2024)
- [Hierarchical Imitation Learning of Team Behavior from Heterogeneous Demonstrations](https://mlsanity.com/paper/2502.17618/) (2025)
- [SEAL: SEmantic-Augmented Imitation Learning via Language Model](https://mlsanity.com/paper/2410.02231/) (2024)
- [Reachability Weighted Offline Goal-conditioned Resampling](https://mlsanity.com/paper/2506.02577/) (2025)
- [Intention-aware Hierarchical Diffusion Model for Long-term Trajectory Anomaly Detection](https://mlsanity.com/paper/2509.17068/) (2025)

## Abstract (arXiv)

Offline imitation learning (IL) refers to learning expert behavior solely from demonstrations, without any additional interaction with the environment. Despite significant advances in offline IL, existing techniques find it challenging to learn policies for long-horizon tasks and require significant re-training when task specifications change. Towards addressing these limitations, we present GO-DICE an offline IL technique for goal-conditioned long-horizon sequential tasks. GO-DICE discerns a hierarchy of sub-tasks from demonstrations and uses these to learn separate policies for sub-task transitions and action execution, respectively; this hierarchical policy learning facilitates long-horizon reasoning. Inspired by the expansive DICE-family of techniques, policy learning at both the levels transpires within the space of stationary distributions. Further, both policies are learnt with goal conditioning to minimize need for retraining when task goals change. Experimental results substantiate that GO-DICE outperforms recent baselines, as evidenced by a marked improvement in the completion rate of increasingly challenging pick-and-place Mujoco robotic tasks. GO-DICE is also capable of leveraging imperfect demonstration and partial task segmentation when available, both of which boost task performance relative to learning from expert demonstrations alone.