mlsanityData to 7 Oct 2026

Imitation from Observations with Trajectory-Level Generative Embeddings

arXiv 2601.00452 ↗First posted 1 Jan 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We consider the offline imitation learning from observations (LfO) where the expert demonstrations are scarce and the available offline suboptimal data are far from the expert behavior. Many existing distribution-matching approaches struggle in this regime because they impose strict support constraints and rely on brittle one-step models, making it hard to extract useful signal from imperfect data. To tackle this challenge, we propose TGE, a trajectory-level generative embedding for offline LfO that constructs a dense, smooth surrogate reward by estimating expert state density in the latent space of a temporal diffusion model trained on offline trajectory data. By leveraging the smooth geometry of the learned diffusion embedding, TGE captures long-horizon temporal dynamics and effectively bridges the gap between disjoint supports, ensuring a robust learning signal even when offline data is distributionally distinct from the expert. Empirically, the proposed approach consistently matches or outperforms prior offline LfO methods across a range of D4RL locomotion and manipulation benchmarks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 41 of its 46 references that are in the map. First posted on arXiv 1 Jan 2026.

Builds on

41 of 46 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets13 Jun
  2. 2022Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching4 Feb
  3. 2021Implicit Behavioral Cloning1 Sep
  4. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  5. 2020Offline Learning from Demonstrations and Unlabeled Experience27 Nov
  6. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  7. 2018Behavioral Cloning from Observation4 May

Other references · 34

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2018World Modelscited by 1,444
  8. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  9. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  11. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  12. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
22 more
  1. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  2. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  3. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  4. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  5. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  6. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  7. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  8. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  9. 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
  10. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  11. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  12. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  13. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 121
  14. 2017Third-Person Imitation Learningcited by 103
  15. 2024Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimizationcited by 99
  16. 2023Diffusion Model as Representation Learnercited by 95
  17. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusioncited by 56
  18. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  19. 2022LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimationcited by 22
  20. 2020Task-Agnostic Exploration via Policy Gradient of a Non-Parametric State Entropy Estimatecited by 21
  21. 2023Offline Imitation from Observation via Primal Wasserstein State Occupancy Matchingcited by 5
  22. 2025Goal-Driven Reward by Video Diffusion Models for Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Robust Offline Imitation Learning Through State-level Trajectory Stitching
  2. 2025Offline Imitation Learning upon Arbitrary Demonstrations by Pre-Training Dynamics Representations
  3. 2023Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching
  4. 2023A Simple Solution for Offline Imitation from Observations and Examples with Possibly Incomplete Trajectories
  5. 2022Imitation Learning from Observations under Transition Model Disparity
  6. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Data
  7. 2021Off-Policy Imitation Learning from Observations
  8. 2024Diffusion Imitation from Observation