mlsanityData to 7 Oct 2026

Decision Transformer under Random Frame Dropping

arXiv 2303.03391 ↗First posted 3 Mar 2023cs.LG cs.AI cs.ROcited by 10

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Controlling agents remotely with deep reinforcement learning~(DRL) in the real world is yet to come. One crucial stepping stone is to devise RL algorithms that are robust in the face of dropped information from corrupted communication or malfunctioning sensors. Typical RL methods usually require considerable online interaction data that are costly and unsafe to collect in the real world. Furthermore, when applying to the frame dropping scenarios, they perform unsatisfactorily even with moderate drop rates. To address these issues, we propose Decision Transformer under Random Frame Dropping~(DeFog), an offline RL algorithm that enables agents to act robustly in frame dropping scenarios without online interaction. DeFog first randomly masks out data in the offline datasets and explicitly adds the time span of frame dropping as inputs. After that, a finetuning stage on the same offline dataset with a higher mask rate would further boost the performance. Empirical results show that DeFog outperforms strong baselines under severe frame drop rates like 90%, while maintaining similar returns under non-frame-dropping conditions in the regular MuJoCo control benchmarks and the Atari environments. Our approach offers a robust and deployable solution for controlling agents in real-world environments with limited or unreliable data.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 35 references that are in the map. First posted on arXiv 3 Mar 2023.

Builds on

19 of 35 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Prompting Decision Transformer for Few-Shot Policy Generalization27 Jun
  2. 2022Multi-Game Decision Transformers30 May
  3. 2022Online Decision Transformer11 Feb
  4. 2022Can Wikipedia Help Offline Reinforcement Learning?28 Jan
  5. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 13

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2022Visual Prompt Tuningcited by 1,674
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  8. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  9. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  10. 2020Reinforcement Learning with Random Delayscited by 56
  11. 2021The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learningcited by 50
  12. 2021Revisiting State Augmentation methods for Reinforcement Learning with Stochastic Delayscited by 20
1 more
  1. 2021Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomizationcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 3; 2025: 2; 2026: 1.

10 papers.

  1. 2023Generalization to New Sequential Decision Making Tasks with In-Context Learningcited by 39
  2. 2023Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learningcited by 24
  3. 2023Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimizationcited by 23
  4. 2024Rethinking Transformers in Solving POMDPscited by 9
  5. 2023Uncertainty-Aware Decision Transformer for Stochastic Driving Environmentscited by 8
  6. 2024Tackling Data Corruption in Offline Reinforcement Learning via Sequence Modelingcited by 7
  7. 2024Multi-objective Cross-task Learning via Goal-conditioned GPT-based Decision Transformers for Surgical Robot Task Automationcited by 4
  8. 2025Accelerating Multi-modal LLM Gaming Performance via Input Prediction and Mishit Correctioncited by 2
  9. 2025Robust In-Context Reinforcement Learning Under Reward Poisoning Attackscited by 0
  10. 2026OnDeFog: Online Decision Transformer under Frame Droppingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Learning Computational Efficient Bots with Costly Features
  2. 2025A Comparison Between Decision Transformers and Traditional Offline Reinforcement Learning Algorithms
  3. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
  4. 2025Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?
  5. 2021The Least Restriction for Offline Reinforcement Learning
  6. 2021Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks
  7. 2022Bridging the Gap Between Offline and Online Reinforcement Learning Evaluation Methodologies
  8. 2025Offline Reinforcement Learning with Penalized Action Noise Injection