# Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making

arXiv: https://arxiv.org/abs/2310.03022 · HTML: https://mlsanity.com/paper/2310.03022/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 31 of its 43 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Placed in

- Offline RL and imitation learning from data / [Decision transformers and sequence modeling](https://mlsanity.com/offlinerl/decision-transformers-and-sequence-modeling/)

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021)
- [MetaFormer Is Actually What You Need for Vision](https://mlsanity.com/paper/2111.11418/) (2021)
- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022)
- [Mastering Atari Games with Limited Data](https://mlsanity.com/paper/2111.00210/) (2021)
- [Online Decision Transformer](https://mlsanity.com/paper/2202.05607/) (2022)
- [RvS: What is Essential for Offline RL via Supervised Learning?](https://mlsanity.com/paper/2112.10751/) (2021)
- [Prompting Decision Transformer for Few-Shot Policy Generalization](https://mlsanity.com/paper/2206.13499/) (2022)
- [Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actions](https://mlsanity.com/paper/1912.02875/) (2019)
- [Policy Expansion for Bridging Offline-to-Online Reinforcement Learning](https://mlsanity.com/paper/2302.00935/) (2023)
- [Reward-Conditioned Policies](https://mlsanity.com/paper/1912.13465/) (2019)
- [Global Context Vision Transformers](https://mlsanity.com/paper/2206.09959/) (2022)
- [Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations](https://mlsanity.com/paper/1606.01305/) (2016)
- [StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learning](https://mlsanity.com/paper/2110.06206/) (2021)
- [Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions](https://mlsanity.com/paper/2303.17396/) (2023)
- [Decision S4: Efficient Sequence-Based RL via State Spaces Layers](https://mlsanity.com/paper/2306.05167/) (2023)

## Built on by (most cited first, 26 of 26)

- [Reinformer: Max-Return Sequence Modeling for Offline RL](https://mlsanity.com/paper/2405.08740/) (2024)
- [Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2403.19925/) (2024)
- [Adaptive Q-Aid for Conditional Supervised Learning in Offline Reinforcement Learning](https://mlsanity.com/paper/2402.02017/) (2024)
- [Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?](https://mlsanity.com/paper/2405.12094/) (2024)
- [Flow Actor-Critic for Offline Reinforcement Learning](https://mlsanity.com/paper/2602.18015/) (2026)
- [Adversarially Robust Decision Transformer](https://mlsanity.com/paper/2407.18414/) (2024)
- [Q-value Regularized Decision ConvFormer for Offline Reinforcement Learning](https://mlsanity.com/paper/2409.08062/) (2024)
- [Return-Aligned Decision Transformer](https://mlsanity.com/paper/2402.03923/) (2024)
- [Unveiling Markov Heads in Pretrained Language Models for Offline Reinforcement Learning](https://mlsanity.com/paper/2409.06985/) (2024)
- [TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models](https://mlsanity.com/paper/2506.12815/) (2025)
- [Mamba as Decision Maker: Exploring Multi-scale Sequence Modeling in Offline Reinforcement Learning](https://mlsanity.com/paper/2406.02013/) (2024)
- [B3C: A Minimalist Approach to Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2501.18138/) (2025)
- [Decision SpikeFormer: Spike-Driven Transformer for Decision Making](https://mlsanity.com/paper/2504.03800/) (2025)
- [Beyond the Known: Decision Making with Counterfactual Reasoning Decision Transformer](https://mlsanity.com/paper/2505.09114/) (2025)
- [Decoupling Return-to-Go for Efficient Decision Transformer](https://mlsanity.com/paper/2601.15953/) (2026)
- [Flow Matching with Injected Noise for Offline-to-Online Reinforcement Learning](https://mlsanity.com/paper/2602.18117/) (2026)
- [Decision MetaMamba: Enhancing Selective SSM in Offline RL with Heterogeneous Sequence Mixing](https://mlsanity.com/paper/2408.10517/) (2024)
- [Decision Transformer vs. Decision Mamba: Analysing the Complexity of Sequential Decision Making in Atari Games](https://mlsanity.com/paper/2412.00725/) (2024)
- [Target Return Optimizer for Multi-Game Decision Transformer](https://mlsanity.com/paper/2503.02311/) (2025)
- [Mildly Conservative Regularized Evaluation for Offline Reinforcement Learning](https://mlsanity.com/paper/2508.05960/) (2025)
- [Unified token representations for sequential decision models](https://mlsanity.com/paper/2510.21448/) (2025)
- [GAS: Enhancing Reward-Cost Balance of Generative Model-assisted Offline Safe RL](https://mlsanity.com/paper/2602.05323/) (2026)
- [Decision MetaMamba: Enhancing Selective SSM in Offline RL with Heterogeneous Sequence Mixing](https://mlsanity.com/paper/2602.19805/) (2026)
- [Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformer](https://mlsanity.com/paper/2605.06104/) (2026)
- [Return-to-Go Is More Than a Number: Q-Guided Alignment for Return-Conditioned Supervised Learning](https://mlsanity.com/paper/2605.29028/) (2026)
- [Trust Guided Decision Transformer](https://mlsanity.com/paper/2609.31586/) (2026)

## Abstract (arXiv)

The recent success of Transformer in natural language processing has sparked its use in various domains. In offline reinforcement learning (RL), Decision Transformer (DT) is emerging as a promising model based on Transformer. However, we discovered that the attention module of DT is not appropriate to capture the inherent local dependence pattern in trajectories of RL modeled as a Markov decision process. To overcome the limitations of DT, we propose a novel action sequence predictor, named Decision ConvFormer (DC), based on the architecture of MetaFormer, which is a general structure to process multiple entities in parallel and understand the interrelationship among the multiple entities. DC employs local convolution filtering as the token mixer and can effectively capture the inherent local associations of the RL dataset. In extensive experiments, DC achieved state-of-the-art performance across various standard RL benchmarks while requiring fewer resources. Furthermore, we show that DC better understands the underlying meaning in data and exhibits enhanced generalization capability.