mlsanityData to 14 Oct 2026

Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloning

arXiv 2506.10137 ↗First posted 11 Jun 2025cs.LG cs.AIcited by 12

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

While goal-conditioned behavior cloning (GCBC) methods can perform well on in-distribution training tasks, they do not necessarily generalize zero-shot to tasks that require conditioning on novel state-goal pairs, i.e. combinatorial generalization. In part, this limitation can be attributed to a lack of temporal consistency in the state representation learned by BC; if temporally correlated states are properly encoded to similar latent representations, then the out-of-distribution gap for novel state-goal pairs would be reduced. We formalize this notion by demonstrating how encouraging long-range temporal consistency via successor representations (SR) can facilitate generalization. We then propose a simple yet effective representation learning objective, BYOL-γ for GCBC, which theoretically approximates the successor representation in the finite MDP case through self-predictive representations, and achieves competitive empirical performance across a suite of challenging tasks requiring combinatorial generalization.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 54 of its 60 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

54 of 60 references.

References · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  5. 2015Continuous control with deep reinforcement learningcited by 3,407
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  9. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  10. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  11. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  12. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
38 more
  1. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  2. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  3. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  4. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  5. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  6. 2021Mastering Atari Games with Limited Datacited by 237
  7. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 218
  8. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  9. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 185
  10. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  11. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 125
  12. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  13. 2021Learning One Representation to Optimize All Rewardscited by 115
  14. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RLcited by 103
  15. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 102
  16. 2023Synthetic Experience Replaycited by 101
  17. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 89
  18. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 83
  19. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 76
  20. 2024Learning and Leveraging World Models in Visual Representation Learningcited by 72
  21. 2021Learning Successor States and Goal-Dependent Values: A Mathematical Viewpointcited by 70
  22. 2025Diffusion Guidance Is a Controllable Policy Improvement Operatorcited by 58
  23. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Makingcited by 56
  24. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 56
  25. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 54
  26. 2022Understanding Self-Predictive Learning for Reinforcement Learningcited by 44
  27. 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 44
  28. 2023Proto-Value Networks: Scaling Representation Learning with Auxiliary Taskscited by 42
  29. 2025Generative Trajectory Stitching through Diffusion Compositioncited by 41
  30. 2018Combined Reinforcement Learning via Abstract Representationscited by 37
  31. 2022On the Generalization of Representations in Reinforcement Learningcited by 28
  32. 2024Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of Viewcited by 27
  33. 2022Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddingscited by 24
  34. 2024GTA: Generative Trajectory Augmentation with Guidance for Offline Reinforcement Learningcited by 22
  35. 2025Horizon Generalization in Reinforcement Learningcited by 17
  36. 2025General agents contain world modelscited by 17
  37. 2024When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learningcited by 14
  38. 2025Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Followingcited by 14

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 10.

12 papers.

  1. 2025TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learningcited by 22
  2. 2025Dual Goal Representationscited by 16
  3. 2026Can We Really Learn One Representation to Optimize All Rewards?cited by 3
  4. 2026Three Steps at a Time: Learning Representations from Action Sequences in Contrastive RLcited by 3
  5. 2026Action-Sufficient Goal Representationscited by 1
  6. 2026Mollified Value Learningcited by 1
  7. 2026Improving Zero-Shot Offline RL via Behavioral Task Samplingcited by 1
  8. 2026MSPR: Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learningcited by 1
  9. 2026Behavioral Foundation Models for Quality Diversitycited by 0
  10. 2026Sparse-TD-JEPA: Improving Zero-Shot Reinforcement Learning With Sparse Latent Task Representationscited by 0
  11. 2026World Modeling through Spectral Alignmentcited by 0
  12. 2026Imagining Goals with Geometric Horizon Models for Policy Generalizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulation
  2. 2026Self-Supervised Keyframe Discovery for Horizon-Invariant Behavior Cloning
  3. 2026Decoupled Behavioral Cloning for Scalable Inductive Generalization in RL from Specifications
  4. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning
  5. 2023Self-Supervised Adversarial Imitation Learning
  6. 2024Constrained Behavior Cloning for Robotic Learning
  7. 2020Imitating Unknown Policies via Exploration
  8. 2024Stable-BC: Controlling Covariate Shift with Stable Behavior Cloning