mlsanityData to 7 Oct 2026

π2vec: Policy Representations with Successor Features

arXiv 2306.09800 ↗First posted 16 Jun 2023cs.LG cs.ROcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper describes π2vec, a method for representing behaviors of black box policies as feature vectors. The policy representations capture how the statistics of foundation model features change in response to the policy behavior in a task agnostic way, and can be trained from offline data, allowing them to be used in offline policy selection. This work provides a key piece of a recipe for fusing together three modern lines of research: Offline policy evaluation as a counterpart to offline RL, foundation models as generic and powerful state representations, and efficient policy selection in resource constrained environments.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 41 references that are in the map. First posted on arXiv 16 Jun 2023.

Builds on

32 of 41 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  8. 2022A Generalist Agentcited by 802
  9. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  10. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  12. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
20 more
  1. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  2. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  3. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  4. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  5. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  6. 2019Batch Policy Learning under Constraintscited by 242
  7. 2018Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimationcited by 226
  8. 2023Vision-Language Models as Success Detectorscited by 138
  9. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 113
  10. 2015An Emphatic Approach to the Problem of Off-policy Temporal-Difference Learningcited by 103
  11. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  12. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 70
  13. 2019Learning When-to-Treat Policiescited by 49
  14. 2019Off-Policy Evaluation via Off-Policy Classificationcited by 32
  15. 2021Autoregressive Dynamics Models for Offline Policy Evaluation and Optimizationcited by 32
  16. 2023Lossless Adaptation of Pretrained Vision Models For Robotic Manipulationcited by 28
  17. 2021Active Offline Policy Selectioncited by 26
  18. 2022Learning Bellman Complete Representations for Offline Policy Evaluationcited by 17
  19. 2021Successor Feature Sets: Generalizing Successor Representations Across Policiescited by 6
  20. 2021Successor Feature Representationscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Lever: Inference-Time Policy Reuse under Support Constraintscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Behavior Prior Representation learning for Offline Reinforcement Learning
  2. 2021Representation Matters: Offline Pretraining for Sequential Decision Making
  3. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directions
  4. 2024Disentangling Policy from Offline Task Representation Learning via Adversarial Data Augmentation
  5. 2024Ensemble Successor Representations for Task Generalization in Offline-to-Online Reinforcement Learning
  6. 2025State Diversity Matters in Offline Behavior Distillation
  7. 2024Finer Behavioral Foundation Models via Auto-Regressive Features and Advantage Weighting
  8. 2025Offline Behavioral Data Selection