mlsanityData to 14 Oct 2026

Masked Trajectory Models for Prediction, Representation, and Control

arXiv 2305.02968 ↗First posted 4 May 2023cs.LG cs.AIcited by 42

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

We introduce Masked Trajectory Models (MTM) as a generic abstraction for sequential decision making. MTM takes a trajectory, such as a state-action sequence, and aims to reconstruct the trajectory conditioned on random subsets of the same trajectory. By training with a highly randomized masking pattern, MTM learns versatile networks that can take on different roles or capabilities, by simply choosing appropriate masks at inference time. For example, the same MTM network can be used as a forward dynamics model, inverse dynamics model, or even an offline RL agent. Through extensive experiments in several continuous control tasks, we show that the same MTM network -- i.e. same weights -- can match or outperform specialized networks trained for the aforementioned capabilities. Additionally, we find that state representations learned by MTM can significantly accelerate the learning speed of traditional RL algorithms. Finally, in offline RL benchmarks, we find that MTM is competitive with specialized offline RL algorithms, despite MTM being a generic self-supervised learning method without any explicit RL components. Code is available at https://github.com/facebookresearch/mtm

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 47 of its 66 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

47 of 66 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Masked Autoencoding for Scalable and Generalizable Decision Making23 Nov
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2016Layer Normalizationcited by 5,669
  7. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  8. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  11. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  12. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
33 more
  1. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  2. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  4. 2020Mastering Atari with Discrete World Modelscited by 1,007
  5. 2018Exploration by Random Network Distillationcited by 993
  6. 2022A Generalist Agentcited by 808
  7. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  8. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  9. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  10. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 657
  11. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  12. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
  13. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  14. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 370
  15. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  16. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  17. 2022Masked Visual Pre-training for Motor Controlcited by 248
  18. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 185
  19. 2021Reinforcement Learning with Prototypical Representationscited by 179
  20. 2022Masked World Models for Visual Controlcited by 170
  21. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  22. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 139
  23. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 125
  24. 2019Training Agents using Upside-Down Reinforcement Learningcited by 107
  25. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learningcited by 96
  26. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 91
  27. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 91
  28. 2020A Game Theoretic Framework for Model Based Reinforcement Learningcited by 73
  29. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 62
  30. 2022Efficient Planning in a Compact Latent Action Spacecited by 57
  31. 2022Semi-Supervised Offline Reinforcement Learning with Action-Free Trajectoriescited by 28
  32. 2022UniMASK: Unified Inference in Sequential Decision Problemscited by 24
  33. 2022Policy Architectures for Compositional Generalization in Controlcited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 13; 2025: 10; 2026: 6.

42 papers.

Sort:

By area:

  1. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  2. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  3. 2025Unified Video Action Modelcited by 205
  4. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 148
  5. 2024Foundation Policies with Hilbert Representationscited by 94
  6. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 61
  7. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 56
  8. 2023PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Playcited by 50
  9. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  10. 2024From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Controlcited by 35
  11. 2023Emergent Agentic Transformer from Chain of Hindsight Experiencecited by 33
  12. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
30 more
  1. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 31
  2. 2025Latent Diffusion Planning for Imitation Learningcited by 26
  3. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioningcited by 25
  4. 2024Learning from Imperfect Demonstrations with Self-Supervision for Robotic Manipulationcited by 14
  5. 2023Goal-Conditioned Predictive Coding for Offline Reinforcement Learningcited by 10
  6. 2023Emergence of Abstract State Representations in Embodied Sequence Modelingcited by 9
  7. 2024Position: Foundation Agents as the Paradigm Shift for Decision Makingcited by 9
  8. 2025ACT-JEPA: Novel Joint-Embedding Predictive Architecture for Efficient Policy Representation Learningcited by 8
  9. 2023PASTA: Pretrained Action-State Transformer Agentscited by 6
  10. 2024Predictive Coding for Decision Transformercited by 6
  11. 2023Masked Path Modeling for Vision-and-Language Navigationcited by 4
  12. 2024Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learningcited by 4
  13. 2025Multimodal Diffusion Forcing for Forceful Manipulationcited by 4
  14. 2024Unveiling Markov Heads in Pretrained Language Models for Offline Reinforcement Learningcited by 3
  15. 2024Learning Versatile Skills with Curriculum Maskingcited by 3
  16. 2025Task Tokens: A Flexible Approach to Adapting Behavior Foundation Modelscited by 3
  17. 2023Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challengescited by 2
  18. 2026Bridging MARL to SARL: An Order-Independent Multi-Agent Transformer via Latent Consensuscited by 2
  19. 2023Enhancing Trajectory Prediction through Self-Supervised Waypoint Noise Predictioncited by 1
  20. 2025Pretraining a Shared Q-Network for Data-Efficient Offline Reinforcement Learningcited by 1
  21. 2024Semi-Supervised One-Shot Imitation Learningcited by 0
  22. 2024M3PC: Test-time Model Predictive Control for Pretrained Masked Trajectory Modelcited by 0
  23. 2025Hybrid Sequence Modeling and Reinforced Verification for Controllable Target-Conditioned Decision Makingcited by 0
  24. 2025GPS-MTM: Capturing Pattern of Normalcy in GPS-Trajectories with self-supervised learningcited by 0
  25. 2025Planning as Descent: Goal-Conditioned Latent Trajectory Synthesis in Learned Energy Landscapescited by 0
  26. 2026Improving and Accelerating Offline RL in Large Discrete Action Spaces with Structured Policy Initializationcited by 0
  27. 2026Tactile Memory with Soft Robot: Robust Object Insertion via Masked Encoding and Soft Wristcited by 0
  28. 2026Beyond Single-Episode Optimization: Sliding-Window Aware Generative Auto-Bidding for Long-Term Advertising Effectivenesscited by 0
  29. 2026PermVLA: Factorization Order as a Regularizer for VLA Learningcited by 0
  30. 2026Looking Ahead and Back: Anticipatory Generative Modeling for Road-Network Trajectoriescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Explicit Trajectory Diversity for RL-Based Post-Training of LLM Agents
  2. 2026TABES: Trajectory-Aware Backward-on-Entropy Steering for Masked Diffusion Models
  3. 2026SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing
  4. 2026TransMASK: Masked State Representation through Learned Transformation
  5. 2025Reinforcement Learning meets Masked Video Modeling : Trajectory-Guided Adaptive Token Selection
  6. 2025Masked Generative Policy for Robotic Control
  7. 2024Excluding the Irrelevant: Focusing Reinforcement Learning through Continuous Action Masking
  8. 2026Divergent Paths: Learning an Outcome-Predictive Representation of LLM Agent Trajectories