mlsanityData to 14 Oct 2026

Masked Autoencoding for Scalable and Generalizable Decision Making

arXiv 2211.12740 ↗First posted 23 Nov 2022cs.LG cs.AIcited by 45

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

We are interested in learning scalable agents for reinforcement learning that can learn from large-scale, diverse sequential data similar to current large vision and language models. To this end, this paper presents masked decision prediction (MaskDP), a simple and scalable self-supervised pretraining method for reinforcement learning (RL) and behavioral cloning (BC). In our MaskDP approach, we employ a masked autoencoder (MAE) to state-action trajectories, wherein we randomly mask state and action tokens and reconstruct the missing data. By doing so, the model is required to infer masked-out states and actions and extract information about dynamics. We find that masking different proportions of the input sequence significantly helps with learning a better model that generalizes well to multiple downstream tasks. In our empirical study, we find that a MaskDP model gains the capability of zero-shot transfer to new BC tasks, such as single and multiple goal reaching, and it can zero-shot infer skills from a few example transitions. In addition, MaskDP transfers well to offline RL and shows promising scaling behavior w.r.t. to model size. It is amenable to data-efficient finetuning, achieving competitive results with prior methods based on autoregressive pretraining.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 35 of its 41 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

35 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Online Decision Transformer11 Feb
  2. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2017Neural Discrete Representation Learningcited by 4,712
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  12. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
20 more
  1. 2018DeepMind Control Suitecited by 956
  2. 2018Behavioral Cloning from Observationcited by 492
  3. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  4. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  5. 2022Masked Visual Pre-training for Motor Controlcited by 248
  6. 2021Reinforcement Learning with Prototypical Representationscited by 179
  7. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
  8. 2022Masked World Models for Visual Controlcited by 170
  9. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 139
  10. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 127
  11. 2021APS: Active Pretraining with Successor Featurescited by 112
  12. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
  13. 2019Scene Memory Transformer for Embodied Agents in Long-Horizon Taskscited by 107
  14. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  15. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learningcited by 96
  16. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 68
  17. 2020VD-BERT: A Unified Vision and Dialog Transformer with BERTcited by 55
  18. 2016Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Modelcited by 54
  19. 2022Generative Pretraining for Black-Box Optimizationcited by 36
  20. 2022UniMASK: Unified Inference in Sequential Decision Problemscited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 18; 2024: 14; 2025: 7; 2026: 5.

45 papers.

Sort:

By area:

  1. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  2. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  3. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  4. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  5. 2025Unified Video Action Modelcited by 205
  6. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  7. 2024Foundation Policies with Hilbert Representationscited by 94
  8. 2023Diffusion Models for Black-Box Optimizationcited by 76
  9. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 56
  10. 2023Masked Trajectory Models for Prediction, Representation, and Controlcited by 42
  11. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 36
  12. 2024From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Controlcited by 35
33 more
  1. 2023Emergent Agentic Transformer from Chain of Hindsight Experiencecited by 33
  2. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  3. 2023Learning to Modulate pre-trained Models in RLcited by 28
  4. 2023DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observabilitycited by 22
  5. 2024Learning Latent Dynamic Robust Representations for World Modelscited by 21
  6. 2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learningcited by 21
  7. 2023Chain-of-Thought Predictive Controlcited by 18
  8. 2025Generative Models in Decision Making: A Surveycited by 15
  9. 2023Goal-Conditioned Predictive Coding for Offline Reinforcement Learningcited by 10
  10. 2024Position: Foundation Agents as the Paradigm Shift for Decision Makingcited by 9
  11. 2023Decision Stacks: Flexible Reinforcement Learning via Modular Generative Modelscited by 8
  12. 2023PASTA: Pretrained Action-State Transformer Agentscited by 6
  13. 2024Predictive Coding for Decision Transformercited by 6
  14. 2024MaxMI: A Maximal Mutual Information Criterion for Manipulation Concept Discoverycited by 5
  15. 2023FoMo Rewards: Can we cast foundation models as reward functions?cited by 4
  16. 2025Multimodal Diffusion Forcing for Forceful Manipulationcited by 4
  17. 2023Aerial Image Object Detection With Vision Transformer Detector (ViTDet)cited by 3
  18. 2023Sequential Action-Induced Invariant Representation for Reinforcement Learningcited by 3
  19. 2023MaskMA: Towards Zero-Shot Multi-Agent Decision Making with Mask-Based Collaborative Learningcited by 3
  20. 2024InfoCon: Concept Discovery with Generative and Discriminative Informativenesscited by 3
  21. 2024Astra: Efficient Transformer Architecture and Contrastive Dynamics Learning for Embodied Instruction Followingcited by 3
  22. 2024Unveiling Markov Heads in Pretrained Language Models for Offline Reinforcement Learningcited by 3
  23. 2024Learning Versatile Skills with Curriculum Maskingcited by 3
  24. 2023Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challengescited by 2
  25. 2026Bridging MARL to SARL: An Order-Independent Multi-Agent Transformer via Latent Consensuscited by 2
  26. 2025Learning Fused State Representations for Control from Multi-View Observationscited by 1
  27. 2024Streetwise Agents: Empowering Offline RL Policies to Outsmart Exogenous Stochastic Disturbances in RTCcited by 0
  28. 2024M3PC: Test-time Model Predictive Control for Pretrained Masked Trajectory Modelcited by 0
  29. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advancescited by 0
  30. 2026Improving and Accelerating Offline RL in Large Discrete Action Spaces with Structured Policy Initializationcited by 0
  31. 2026Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAscited by 0
  32. 2026Mask-based Predictive Representations for Reinforcement Learningcited by 0
  33. 2026PermVLA: Factorization Order as a Regularizer for VLA Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning
  2. 2024Excluding the Irrelevant: Focusing Reinforcement Learning through Continuous Action Masking
  3. 2025MaskFocus: Focusing Policy Optimization on Critical Steps for Masked Image Generation
  4. 2025Reinforcement Learning meets Masked Video Modeling : Trajectory-Guided Adaptive Token Selection
  5. 2026Neuro-symbolic Action Masking for Deep Reinforcement Learning
  6. 2025Masked Generative Policy for Robotic Control
  7. 2025Masked Auto-Regressive Variational Acceleration: Fast Inference Makes Practical Reinforcement Learning
  8. 2026VAM: Verbalized Action Masking for Controllable Exploration in RL Post-Training -- A Chess Case Study