mlsanityData to 14 Oct 2026

Context Representation via Action-Free Transformer encoder-decoder for Meta Reinforcement Learning

arXiv 2512.14057 ↗First posted 16 Dec 2025cs.ROcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement learning (RL) enables robots to operate in uncertain environments, but standard approaches often struggle with poor generalization to unseen tasks. Context-adaptive meta reinforcement learning addresses these limitations by conditioning on the task representation, yet they mostly rely on complete action information in the experience making task inference tightly coupled to a specific policy. This paper introduces Context Representation via Action Free Transformer encoder decoder (CRAFT), a belief model that infers task representations solely from sequences of states and rewards. By removing the dependence on actions, CRAFT decouples task inference from policy optimization, supports modular training, and leverages amortized variational inference for scalable belief updates. Built on a transformer encoder decoder with rotary positional embeddings, the model captures long range temporal dependencies and robustly encodes both parametric and non-parametric task variations. Experiments on the MetaWorld ML-10 robotic manipulation benchmark show that CRAFT achieves faster adaptation, improved generalization, and more effective exploration compared to context adaptive meta--RL baselines. These findings highlight the potential of action-free inference as a foundation for scalable RL in robotic control.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 33 references that are in the map. First posted on arXiv 16 Dec 2025.

Builds on

24 of 33 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Meta-World+: An Improved, Standardized, RL Benchmark16 May
  2. 2024Hierarchical Transformers are Efficient Meta-Reinforcement Learners9 Feb

Other references · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  8. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  9. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  10. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 228
  11. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  12. 2022Online Decision Transformercited by 196
10 more
  1. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 146
  2. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
  3. 2019Meta reinforcement learning as task inferencecited by 77
  4. 2019Meta-learning of Sequential Strategiescited by 70
  5. 2022Hypernetworks in Meta-Reinforcement Learningcited by 42
  6. 2022Transformers are Meta-Reinforcement Learnerscited by 41
  7. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
  8. 2022Skill-based Meta-Reinforcement Learningcited by 37
  9. 2022Semi-Supervised Offline Reinforcement Learning with Action-Free Trajectoriescited by 28
  10. 2021Meta-Reinforcement Learning in Broad and Non-Parametric Environmentscited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Behavior-Invariant Task Representation Learning with Transformer-based World Models for Offline Meta-Reinforcement Learning
  2. 2023Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skills
  3. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglement
  4. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers
  5. 2026Robotic Policy Adaptation via Weight-Space Meta-Learning
  6. 2025Disentangled Multi-Context Meta-Learning: Unlocking robust and Generalized Task Learning
  7. 2024Scrutinize What We Ignore: Reining In Task Representation Shift Of Context-Based Offline Meta Reinforcement Learning
  8. 2025Heuristic Transformer: Belief Augmented In-Context Reinforcement Learning