mlsanityData to 14 Oct 2026

Contextual Intelligence The Next Leap for Reinforcement Learning

arXiv 2604.02348 ↗First posted 17 Feb 2026cs.LGcited by 1

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement learning (RL) has produced spectacular results in games, robotics, and continuous control. Yet, despite these successes, learned policies often fail to generalize beyond their training distribution, limiting real-world impact. Recent work on contextual RL (cRL) shows that exposing agents to environment characteristics -- contexts -- can improve zero-shot transfer. So far, the community has treated context as a monolithic, static observable, an approach that constrains the generalization capabilities of RL agents. To achieve contextual intelligence we first propose a novel taxonomy of contexts that separates allogenic (environment-imposed) from autogenic (agent-driven) factors. We identify three fundamental research directions that must be addressed to promote truly contextual intelligence: (1) Learning with heterogeneous contexts to explicitly exploit the taxonomy levels so agents can reason about their influence on the world and vice versa; (2) Multi-time-scale modeling to recognize that allogenic variables evolve slowly or remain static, whereas autogenic variables may change within an episode, potentially requiring different learning mechanisms; (3) Integration of abstract, high-level contexts to incorporate roles, resource & regulatory regimes, uncertainties, and other non-physical descriptors that crucially influence behavior. We envision context as a first-class modeling primitive, empowering agents to reason about who they are, what the world permits, and how both evolve over time. By doing so, we aim to catalyze a new generation of context-aware agents that can be deployed safely and efficiently in the real world.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 32 of its 51 references that are in the map. First posted on arXiv 17 Feb 2026.

Builds on

32 of 51 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Learning to Explore in Diverse Reward Settings via Temporal-Difference-Error Maximization16 Jun
  2. 2024Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learning15 Apr

Other references · 30

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  3. 2016HyperNetworkscited by 1,027
  4. 2018Learning Dexterous In-Hand Manipulationcited by 687
  5. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  6. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  7. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
  8. 2015Contextual Markov Decision Processescited by 220
  9. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 194
  10. 2021Reinforcement Learning with Prototypical Representationscited by 179
  11. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  12. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
18 more
  1. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109
  2. 2017Markov Decision Processes with Continuous Side Informationcited by 56
  3. 2020Self-Paced Deep Reinforcement Learningcited by 50
  4. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 46
  5. 2021Clockwork Variational Autoencoderscited by 42
  6. 2019Environment Probing Interaction Policiescited by 39
  7. 2022Context is Everything: Implicit Identification for Dynamics Adaptationcited by 27
  8. 2021Self-Paced Context Evaluation for Contextual Reinforcement Learningcited by 25
  9. 2021TempoRL: Learning When to Actcited by 25
  10. 2020Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trialscited by 20
  11. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 20
  12. 2024Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalizationcited by 16
  13. 2021A New Representation of Successor Features for Transfer across Dissimilar Environmentscited by 13
  14. 2025Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policycited by 5
  15. 2022Hyperparameters in Contextual RL are Highly Situationalcited by 4
  16. 2024CANDID DAC: Leveraging Coupled Action Dimensions with Importance Differences in DACcited by 4
  17. 2025Can Context Bridge the Reality Gap? Sim-to-Real Transfer of Context-Aware Policiescited by 4
  18. 2025Shared Modular Recurrence in Contextual MDPs for Universal Morphology Controlcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Dynamic Context Scheduling: Learning Beyond the Static Universecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Considering Context: When World Models Need Context Encoders
  2. 2026In-Context Reinforcement Learning under Non-Stationarity: A Survey
  3. 2026From Pixels to Digital Agents: An Empirical Study on the Taxonomy and Technological Trends of Reinforcement Learning Environments
  4. 2023Foundations for Transfer in Reinforcement Learning: A Taxonomy of Knowledge Modalities
  5. 2025Towards Provable Emergence of In-Context Reinforcement Learning
  6. 2025In-Context Reinforcement Learning via Communicative World Models
  7. 2025Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts
  8. 2024Towards Generalizable Reinforcement Learning via Causality-Guided Self-Adaptive Representations