mlsanityData to 7 Oct 2026

Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making

arXiv 2310.03022 ↗First posted 4 Oct 2023cs.LGcited by 26

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

The recent success of Transformer in natural language processing has sparked its use in various domains. In offline reinforcement learning (RL), Decision Transformer (DT) is emerging as a promising model based on Transformer. However, we discovered that the attention module of DT is not appropriate to capture the inherent local dependence pattern in trajectories of RL modeled as a Markov decision process. To overcome the limitations of DT, we propose a novel action sequence predictor, named Decision ConvFormer (DC), based on the architecture of MetaFormer, which is a general structure to process multiple entities in parallel and understand the interrelationship among the multiple entities. DC employs local convolution filtering as the token mixer and can effectively capture the inherent local associations of the RL dataset. In extensive experiments, DC achieved state-of-the-art performance across various standard RL benchmarks while requiring fewer resources. Furthermore, we show that DC better understands the underlying meaning in data and exhibits enhanced generalization capability.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 43 references that are in the map. First posted on arXiv 4 Oct 2023.

Builds on

31 of 43 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Decision S4: Efficient Sequence-Based RL via State Spaces Layers8 Jun
  2. 2023Merging Decision Transformers: Weight Averaging for Forming Multi-Task Policies14 Mar
  3. 2022Prompting Decision Transformer for Few-Shot Policy Generalization27 Jun
  4. 2022Online Decision Transformer11 Feb
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  8. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  10. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  11. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  12. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
14 more
  1. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  2. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  3. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  4. 2021MetaFormer Is Actually What You Need for Visioncited by 523
  5. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  6. 2021Mastering Atari Games with Limited Datacited by 235
  7. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  8. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  9. 2023Policy Expansion for Bridging Offline-to-Online Reinforcement Learningcited by 103
  10. 2019Reward-Conditioned Policiescited by 101
  11. 2022Global Context Vision Transformerscited by 98
  12. 2016Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activationscited by 54
  13. 2021StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learningcited by 27
  14. 2023Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutionscited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 7; 2026: 8.

26 papers.

Sort:

By area:

  1. 2024Reinformer: Max-Return Sequence Modeling for Offline RLcited by 25
  2. 2024Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spacescited by 23
  3. 2024Adaptive Q-Aid for Conditional Supervised Learning in Offline Reinforcement Learningcited by 7
  4. 2024Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?cited by 7
  5. 2026Flow Actor-Critic for Offline Reinforcement Learningcited by 6
  6. 2024Adversarially Robust Decision Transformercited by 4
  7. 2024Return-Aligned Decision Transformercited by 3
  8. 2024Unveiling Markov Heads in Pretrained Language Models for Offline Reinforcement Learningcited by 3
  9. 2024Q-value Regularized Decision ConvFormer for Offline Reinforcement Learningcited by 3
  10. 2025TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Modelscited by 2
  11. 2024Mamba as Decision Maker: Exploring Multi-scale Sequence Modeling in Offline Reinforcement Learningcited by 1
  12. 2025B3C: A Minimalist Approach to Offline Multi-Agent Reinforcement Learningcited by 1
14 more
  1. 2025Decision SpikeFormer: Spike-Driven Transformer for Decision Makingcited by 1
  2. 2025Beyond the Known: Decision Making with Counterfactual Reasoning Decision Transformercited by 1
  3. 2026Decoupling Return-to-Go for Efficient Decision Transformercited by 1
  4. 2024Decision MetaMamba: Enhancing Selective SSM in Offline RL with Heterogeneous Sequence Mixingcited by 0
  5. 2024Decision Transformer vs. Decision Mamba: Analysing the Complexity of Sequential Decision Making in Atari Gamescited by 0
  6. 2025Target Return Optimizer for Multi-Game Decision Transformercited by 0
  7. 2025Mildly Conservative Regularized Evaluation for Offline Reinforcement Learningcited by 0
  8. 2025Unified token representations for sequential decision modelscited by 0
  9. 2026GAS: Enhancing Reward-Cost Balance of Generative Model-assisted Offline Safe RLcited by 0
  10. 2026Flow Matching with Injected Noise for Offline-to-Online Reinforcement Learningcited by 0
  11. 2026Decision MetaMamba: Enhancing Selective SSM in Offline RL with Heterogeneous Sequence Mixingcited by 0
  12. 2026Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformercited by 0
  13. 2026Return-to-Go Is More Than a Number: Q-Guided Alignment for Return-Conditioned Supervised Learningcited by 0
  14. 2026Trust Guided Decision Transformercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Pre-trained Language Models Improve the Few-shot Prompt Ability of Decision Transformer
  2. 2023Rethinking Decision Transformer via Hierarchical Reinforcement Learning
  3. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglement
  4. 2025Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?
  5. 2022Contextual Transformer for Offline Meta Reinforcement Learning
  6. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL
  7. 2025Adjusting the Output of Decision Transformer with Action Gradient
  8. 2024In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought