mlsanityData to 7 Oct 2026

Reinforcement Learning from Passive Data via Latent Intentions

arXiv 2304.04782 ↗First posted 10 Apr 2023cs.LG cs.AI stat.MLcited by 64

Not on the map. Why?

Cited by 21 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Passive observational data, such as human videos, is abundant and rich in information, yet remains largely untapped by current RL methods. Perhaps surprisingly, we show that passive data, despite not having reward or action labels, can still be used to learn features that accelerate downstream RL. Our approach learns from passive data by modeling intentions: measuring how the likelihood of future outcomes change when the agent acts to achieve a particular task. We propose a temporal difference learning objective to learn about intentions, resulting in an algorithm similar to conventional RL, but which learns entirely from passive data. When optimizing this objective, our agent simultaneously learns representations of states, of policies, and of possible outcomes in an environment, all from raw observational data. Both theoretically and empirically, this scheme learns features amenable for value prediction for downstream tasks, and our experiments demonstrate the ability to learn from many forms of passive data, including cross-embodiment video data and YouTube videos.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 32 of its 42 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

32 of 42 references.

References · 32

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  5. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  6. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  7. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  8. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  9. 2018Behavioral Cloning from Observationcited by 482
  10. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  11. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  12. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
20 more
  1. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  2. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  3. 2022Masked Visual Pre-training for Motor Controlcited by 246
  4. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  5. 2018Generative Adversarial Imitation from Observationcited by 183
  6. 2022Masked World Models for Visual Controlcited by 169
  7. 2021Learning One Representation to Optimize All Rewardscited by 110
  8. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  9. 2017Third-Person Imitation Learningcited by 103
  10. 2018Universal Successor Features Approximatorscited by 102
  11. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 101
  12. 2017Eigenoption Discovery through the Deep Successor Representationcited by 85
  13. 2020Reinforcement Learning with Videos: Combining Offline Observations with Interactioncited by 80
  14. 2016Deep Successor Reinforcement Learningcited by 72
  15. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environmentscited by 62
  16. 2022Dichotomy of Control: Separating What You Can Control from What You Cannotcited by 47
  17. 2020The MAGICAL Benchmark for Robust Imitationcited by 30
  18. 2022Addressing Optimism Bias in Sequence Modeling for Reinforcement Learningcited by 22
  19. 2019Perceptual Values from Observationcited by 11
  20. 2022Learning Value Functions from Undirected State-only Experiencecited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 24; 2025: 20; 2026: 12.

64 papers.

Sort:

By area:

  1. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 216
  2. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  3. 2023Learning to Act without Actionscited by 116
  4. 2024Foundation Policies with Hilbert Representationscited by 89
  5. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Makingcited by 53
  6. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
  7. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  8. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
  9. 2023Robotic Offline RL from Internet Videos via Value-Function Pre-Trainingcited by 31
  10. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 30
  11. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  12. 2023Accelerating Exploration with Unlabeled Prior Datacited by 21
38 more
  1. 2025Temporal Difference Flowscited by 20
  2. 2025Value-guided action planning with JEPA world modelscited by 19
  3. 2025AMPLIFY: Actionless Motion Priors for Robot Learning from Videoscited by 18
  4. 2024Proto Successor Measure: Representing the Behavior Space of an RL Agentcited by 17
  5. 2025TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learningcited by 17
  6. 2024A Distributional Analogue to the Successor Representationcited by 15
  7. 2025Physics-informed Value Learner for Offline Goal-Conditioned Reinforcement Learningcited by 15
  8. 2023Unsupervised Behavior Extraction via Random Intent Priorscited by 13
  9. 2024Leveraging Skills from Unlabeled Prior Data for Efficient Online Explorationcited by 13
  10. 2025Dual Goal Representationscited by 13
  11. 2024Diffusion Imitation from Observationcited by 12
  12. 2025Transitive RL: Value Learning via Divide and Conquercited by 12
  13. 2024Unsupervised-to-Online Reinforcement Learningcited by 11
  14. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matchingcited by 10
  15. 2024PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learningcited by 9
  16. 2025Object-Centric Latent Action Learningcited by 9
  17. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learningcited by 8
  18. 2025Test-time Offline Reinforcement Learning on Goal-related Experiencecited by 8
  19. 2023Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learningcited by 7
  20. 2024Align Your Intents: Offline Imitation Learning via Optimal Transportcited by 7
  21. 2024Constrained Ensemble Exploration for Unsupervised Skill Discoverycited by 7
  22. 2024On-Robot Reinforcement Learning with Goal-Contrastive Rewardscited by 7
  23. 2025Intention-Conditioned Flow Occupancy Modelscited by 7
  24. 2024ENOT: Expectile Regularization for Fast and Accurate Training of Neural Optimal Transportcited by 6
  25. 2024Predictive Coding for Decision Transformercited by 6
  26. 2025Expressive Value Learning for Scalable Offline Reinforcement Learningcited by 6
  27. 2024Hybrid Reinforcement Learning from Offline Observation Alonecited by 5
  28. 2025Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learningcited by 5
  29. 2026Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RLcited by 5
  30. 2024Video Occupancy Modelscited by 4
  31. 2024Sample-efficient Unsupervised Policy Cloning from Ensemble Self-supervised Labeled Videoscited by 4
  32. 2025Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representationscited by 4
  33. 2024NOLO: Navigate Only Look Oncecited by 3
  34. 2025Spectral Representation-based Reinforcement Learningcited by 3
  35. 2024DiffGen: Robot Demonstration Generation via Differentiable Physics Simulation, Differentiable Rendering, and Vision-Language Modelcited by 2
  36. 2024Goal-Reaching Policy Learning from Non-Expert Observations via Effective Subgoal Guidancecited by 2
  37. 2024Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learningcited by 2
  38. 2025Offline Action-Free Learning of Ex-BMDPs by Comparing Diverse Datasetscited by 2

50 most cited of 64 All 64 (JSON)