mlsanityData to 7 Oct 2026

LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimation

arXiv 2202.13536 ↗First posted 28 Feb 2022cs.LG cs.AIcited by 22

Not on the map. Why?

Abstract (arXiv)

We consider the problem of learning from observation (LfO), in which the agent aims to mimic the expert's behavior from the state-only demonstrations by experts. We additionally assume that the agent cannot interact with the environment but has access to the action-labeled transition data collected by some agents with unknown qualities. This offline setting for LfO is appealing in many real-world scenarios where the ground-truth expert actions are inaccessible and the arbitrary environment interactions are costly or risky. In this paper, we present LobsDICE, an offline LfO algorithm that learns to imitate the expert policy via optimization in the space of stationary distributions. Our algorithm solves a single convex minimization problem, which minimizes the divergence between the two state-transition distributions induced by the expert and the agent policy. Through an extensive set of offline LfO tasks, we show that LobsDICE outperforms strong baseline methods.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 31 of its 52 references that are in the map. First posted on arXiv 28 Feb 2022.

Builds on

31 of 52 references.

References · 31

  1. 2017Improved Training of Wasserstein GANscited by 2,223
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2016OpenAI Gymcited by 1,684
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2018Behavioral Cloning from Observationcited by 482
  9. 2020Is Pessimism Provably Efficient for Offline RL?cited by 362
  10. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  11. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  12. 2019AlgaeDICE: Policy Gradient from Arbitrary Experiencecited by 187
19 more
  1. 2018Generative Adversarial Imitation from Observationcited by 183
  2. 2019Imitation Learning via Off-Policy Distribution Matchingcited by 182
  3. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  4. 2017Safe Policy Improvement with Baseline Bootstrappingcited by 141
  5. 2020GenDICE: Generalized Offline Estimation of Stationary Valuescited by 122
  6. 2019Imitation Learning as f-Divergence Minimizationcited by 121
  7. 2021OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimationcited by 121
  8. 2020Off-Policy Evaluation via the Regularized Lagrangiancited by 86
  9. 2019Provably Efficient Imitation Learning from Observation Alonecited by 78
  10. 2021Conservative Offline Distributional Reinforcement Learningcited by 78
  11. 2019State Alignment-based Imitation Learningcited by 73
  12. 2020GradientDICE: Rethinking Generalized Offline Estimation of Stationary Valuescited by 70
  13. 2021Off-Policy Imitation Learning from Observationscited by 70
  14. 2020CoinDICE: Off-Policy Confidence Interval Estimationcited by 64
  15. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
  16. 2021MobILE: Model-Based Imitation Learning From Observation Alonecited by 44
  17. 2021Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classificationcited by 38
  18. 2020Offline Policy Selection under Uncertaintycited by 31
  19. 2022Rethinking ValueDice: Does It Really Improve Performance?cited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 9; 2025: 8; 2026: 2.

22 papers.

Sort:

By area:

  1. 2024SEABO: A Simple Search-Based Method for Offline Imitation Learningcited by 14
  2. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasetscited by 10
  3. 2024SPRINQL: Sub-optimal Demonstrations driven Offline Imitation Learningcited by 7
  4. 2023GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimationcited by 6
  5. 2024OLLIE: Imitation Learning from Offline Pretraining to Online Finetuningcited by 6
  6. 2023Offline Imitation from Observation via Primal Wasserstein State Occupancy Matchingcited by 5
  7. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  8. 2024Offline Hierarchical Reinforcement Learning via Inverse Optimizationcited by 4
  9. 2023A Simple Solution for Offline Imitation from Observations and Examples with Possibly Incomplete Trajectoriescited by 3
  10. 2025Offline Imitation Learning upon Arbitrary Demonstrations by Pre-Training Dynamics Representationscited by 3
  11. 2024Surgical Task Automation Using Actor-Critic Frameworks and Self-Supervised Imitation Learningcited by 2
  12. 2025PROF: An LLM-based Reward Code Preference Optimization Framework for Offline Imitation Learningcited by 2
10 more
  1. 2025Bellman Calibration for V-Learning in Offline Reinforcement Learningcited by 2
  2. 2025SEMDICE: Off-policy State Entropy Maximization via Stationary Distribution Correction Estimationcited by 1
  3. 2024Offline Imitation Learning by Controlling the Effective Planning Horizoncited by 0
  4. 2024Robust off-policy Reinforcement Learning via Soft Constrained Adversarycited by 0
  5. 2024UNIQ: Offline Inverse Q-learning for Avoiding Undesirable Demonstrationscited by 0
  6. 2025Learning What to Do and What Not To Do: Offline Imitation from Expert and Undesirable Demonstrationscited by 0
  7. 2025Off-Policy Actor-Critic for Adversarial Observation Robustness: Virtual Alternative Training via Symmetric Policy Evaluationcited by 0
  8. 2025Learning from Observation: A Survey of Recent Advancescited by 0
  9. 2026Imitation from Observations with Trajectory-Level Generative Embeddingscited by 0
  10. 2026Latent Wasserstein Adversarial Imitation Learningcited by 0