mlsanityData to 7 Oct 2026

Learning Versatile Skills with Curriculum Masking

arXiv 2410.17744 ↗First posted 23 Oct 2024cs.LG cs.AIcited by 2

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Masked prediction has emerged as a promising pretraining paradigm in offline reinforcement learning (RL) due to its versatile masking schemes, enabling flexible inference across various downstream tasks with a unified model. Despite the versatility of masked prediction, it remains unclear how to balance the learning of skills at different levels of complexity. To address this, we propose CurrMask, a curriculum masking pretraining paradigm for sequential decision making. Motivated by how humans learn by organizing knowledge in a curriculum, CurrMask adjusts its masking scheme during pretraining for learning versatile skills. Through extensive experiments, we show that CurrMask exhibits superior zero-shot performance on skill prompting tasks, goal-conditioned planning tasks, and competitive finetuning performance on offline RL tasks. Additionally, our analysis of training dynamics reveals that CurrMask gradually acquires skills of varying complexity by dynamically adjusting its masking scheme.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 43 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

31 of 43 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023PASTA: Pretrained Action-State Transformer Agents20 Jul
  2. 2023Future-conditioned Unsupervised Pretraining for Decision Transformer26 May
  3. 2023Masked Trajectory Models for Prediction, Representation, and Control4 May
  4. 2022Masked Autoencoding for Scalable and Generalizable Decision Making23 Nov

Other references · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  7. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  8. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  9. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  10. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  11. 2017Automated Curriculum Learning for Neural Networkscited by 251
  12. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
15 more
  1. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  2. 2021Reinforcement Learning with Prototypical Representationscited by 179
  3. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  4. 2022Masked World Models for Visual Controlcited by 169
  5. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  6. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  7. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  8. 2022What to Hide from Your Students: Attention-Guided Masked Image Modelingcited by 117
  9. 2021MST: Masked Self-Supervised Transformer for Visual Representationcited by 114
  10. 2022SemMAE: Semantic-Guided Masking for Learning Masked Autoencoderscited by 113
  11. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  12. 2023SMART: Self-supervised Multi-task pretrAining with contRol Transformerscited by 44
  13. 2019Unsupervised Curricula for Visual Meta-Reinforcement Learningcited by 33
  14. 2020Adaptive Procedural Task Generation for Hard-Exploration Problemscited by 20
  15. 2023Ignorance is Bliss: Robust Control via Information Gatingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025RAD: Retrieval High-quality Demonstrations to Enhance Decision-makingcited by 0
  2. 2026Beyond Single-Episode Optimization: Sliding-Window Aware Generative Auto-Bidding for Long-Term Advertising Effectivenesscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022UniMASK: Unified Inference in Sequential Decision Problems
  2. 2021Representation Matters: Offline Pretraining for Sequential Decision Making
  3. 2024Robust Policy Learning via Offline Skill Diffusion
  4. 2024Foundation Policies with Hilbert Representations
  5. 2025Offline Reinforcement Learning with Discrete Diffusion Skills
  6. 2026Control-Diverse Reinforcement Fine-Tuning: Decoupling the Shared Control Bottleneck of RL Post-Training
  7. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
  8. 2022Bayesian Nonparametrics for Offline Skill Discovery