mlsanityData to 7 Oct 2026

Temporal Consistency Improves Generalization in Contextual Offline Meta Reinforcement Learning

arXiv 2603.02935 ↗First posted 3 Mar 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Offline meta-reinforcement learning seeks to learn a policy that generalizes to new related tasks online. Context-based methods infer a task representation from transition histories, yet learning an effective task representation without supervision remains challenging. Existing methods relying on contrastive learning learn discriminative task representations, but fail to identify task-specific dynamics, while relying on reconstruction can be insufficient to model long-horizon dependencies, limiting generalization to new tasks. We investigate the impact of temporal consistency in latent space on task representation learning, showing that enforcing multi-step predictions in latent space encourages task representations that are able to capture task-dependent dynamics while preventing representation collapse. We provide theoretical analysis characterizing sources of error in value estimation and show through extensive experiments on MuJoCo, Contextual DeepMind Control, and MetaWorld benchmarks that temporal consistency significantly improves both zero-shot and few-shot generalization.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 43 of its 56 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

43 of 56 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Entropy Regularized Task Representation Learning for Offline Meta-Reinforcement Learning19 Dec
  2. 2024Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning4 Feb
  3. 2023Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitations26 Dec
  4. 2023Context Shift Reduction for Offline Meta-Reinforcement Learning7 Nov
  5. 2023Offline Meta Reinforcement Learning with In-Distribution Online Adaptation31 May
  6. 2022Prompting Decision Transformer for Few-Shot Policy Generalization27 Jun
  7. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learning21 Jun
  8. 2021Offline Meta-Reinforcement Learning with Online Self-Supervision8 Jul

Other references · 35

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Layer Normalizationcited by 5,579
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  9. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  10. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  11. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  12. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
23 more
  1. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  2. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  3. 2020CLUB: A Contrastive Log-ratio Upper Bound of Mutual Informationcited by 233
  4. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
  5. 2019Mish: A Self Regularized Non-Monotonic Activation Functioncited by 141
  6. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
  7. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124
  8. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108
  9. 2023A Tutorial on Meta-Reinforcement Learningcited by 102
  10. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  11. 2018Probabilistic Recurrent State-Space Modelscited by 64
  12. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
  13. 2023Disentanglement via Latent Quantizationcited by 40
  14. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 40
  15. 2023Sharpness-Aware Minimization Leads to Low-Rank Featurescited by 38
  16. 2024No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPOcited by 27
  17. 2022Hypernetworks for Zero-shot Transfer in Reinforcement Learningcited by 18
  18. 2023Simplified Temporal Consistency Reinforcement Learningcited by 17
  19. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglementcited by 16
  20. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  21. 2024iQRL -- Implicitly Quantized Representations for Sample-efficient Reinforcement Learningcited by 9
  22. 2025Discrete Codebook World Models for Continuous Controlcited by 9
  23. 2024Debiased Offline Representation Learning for Fast Online Adaptation in Non-stationary Dynamicscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Behavior-Invariant Task Representation Learning with Transformer-based World Models for Offline Meta-Reinforcement Learning
  2. 2024Scrutinize What We Ignore: Reining In Task Representation Shift Of Context-Based Offline Meta Reinforcement Learning
  3. 2020FOCAL: Efficient Fully-Offline Meta-Reinforcement Learning via Distance Metric Learning and Behavior Regularization
  4. 2024Disentangling Policy from Offline Task Representation Learning via Adversarial Data Augmentation
  5. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learning
  6. 2022Contextual Transformer for Offline Meta Reinforcement Learning
  7. 2021Provably Improved Context-Based Offline Meta-RL with Attention and Contrastive Learning
  8. 2022Generalization with Lossy Affordances: Leveraging Broad Offline Data for Learning Visuomotor Tasks