mlsanityData to 7 Oct 2026

Predictive representations: building blocks of intelligence

arXiv 2402.06590 ↗First posted 9 Feb 2024cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Adaptive behavior often requires predicting future events. The theory of reinforcement learning prescribes what kinds of predictive representations are useful and how to compute them. This paper integrates these theoretical ideas with work on cognition and neuroscience. We pay special attention to the successor representation (SR) and its generalizations, which have been widely applied both as engineering tools and models of brain function. This convergence suggests that particular kinds of predictive representations may function as versatile building blocks of intelligence.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 42 of its 210 references that are in the map. First posted on arXiv 9 Feb 2024.

Builds on

42 of 210 references.

References · 42

  1. 2017Hindsight Experience Replaycited by 1,096
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  3. 2019The StarCraft Multi-Agent Challengecited by 589
  4. 2019On Variational Bounds of Mutual Informationcited by 550
  5. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  6. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  7. 2016Deep Predictive Coding Networks for Video Prediction and Unsupervised Learningcited by 203
  8. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  9. 2021APS: Active Pretraining with Successor Featurescited by 109
  10. 2018Universal Successor Features Approximatorscited by 102
  11. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 99
  12. 2017Eigenoption Discovery through the Deep Successor Representationcited by 85
30 more
  1. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
  2. 2016Deep Successor Reinforcement Learningcited by 72
  3. 2018Improving Regression Performance with Distributional Lossescited by 66
  4. 2021Temporal Abstraction in Reinforcement Learning with the Successor Representationcited by 52
  5. 2021The Difficulty of Passive Learning in Deep Reinforcement Learningcited by 50
  6. 2021Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planningcited by 43
  7. 2023Proto-Value Networks: Scaling Representation Learning with Auxiliary Taskscited by 41
  8. 2023Contrastive Difference Predictive Codingcited by 35
  9. 2020UneVEn: Universal Value Exploration for Multi-Agent Reinforcement Learningcited by 32
  10. 2019Truly Batch Apprenticeship Learning with Deep Successor Featurescited by 27
  11. 2019Successor Options: An Option Discovery Framework for Reinforcement Learningcited by 23
  12. 2021Successor Feature Landmarks for Long-Horizon Goal-Conditioned Reinforcement Learningcited by 23
  13. 2022Optimistic Linear Support and Successor Features as a Basis for Optimal Policy Transfercited by 22
  14. 2021Discovering Diverse Nearly Optimal Policies with Successor Featurescited by 20
  15. 2021A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representationcited by 19
  16. 2023Diversifying AI: Towards Creative Chess with AlphaZerocited by 15
  17. 2021A First-Occupancy Representation for Reinforcement Learningcited by 14
  18. 2021Constructing a Good Behavior Basis for Transfer using Generalized Policy Updatescited by 14
  19. 2021A New Representation of Successor Features for Transfer across Dissimilar Environmentscited by 13
  20. 2023Combining Behaviors with the Successor Features Keyboardcited by 12
  21. 2022Generalised Policy Improvement with Geometric Policy Compositioncited by 11
  22. 2022Actionable Neural Representations: Grid Cells from Minimal Constraintscited by 10
  23. 2022Continual Auxiliary Task Learningcited by 9
  24. 2023Composing Task Knowledge with Modular Successor Feature Approximatorscited by 7
  25. 2019Successor Features Combine Elements of Model-Free and Model-based Reinforcement Learningcited by 6
  26. 2022USHER: Unbiased Sampling for Hindsight Experience Replaycited by 6
  27. 2023Successor-Predecessor Intrinsic Explorationcited by 6
  28. 2021Successor Feature Neural Episodic Controlcited by 3
  29. 2021Option Transfer and SMDP Abstraction with Successor Featurescited by 1
  30. 2022Disentangling Successor Features for Coordination in Multi-agent Reinforcement Learningcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloningcited by 9
  2. 2025Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environmentscited by 4
  3. 2025Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machinescited by 2
  4. 2025Goals and the Structure of Experiencecited by 0