mlsanityData to 7 Oct 2026

Prompting Decision Transformer for Few-Shot Policy Generalization

arXiv 2206.13499 ↗First posted 27 Jun 2022cs.LG cs.AI cs.CVcited by 143

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Humans can leverage prior experience and learn novel tasks from a handful of demonstrations. In contrast to offline meta-reinforcement learning, which aims to achieve quick adaptation through better algorithm design, we investigate the effect of architecture inductive bias on the few-shot learning capability. We propose a Prompt-based Decision Transformer (Prompt-DT), which leverages the sequential modeling ability of the Transformer architecture and the prompt framework to achieve few-shot adaptation in offline RL. We design the trajectory prompt, which contains segments of the few-shot demonstrations, and encodes task-specific information to guide policy generation. Our experiments in five MuJoCo control benchmarks show that Prompt-DT is a strong few-shot learner without any extra finetuning on unseen target tasks. Prompt-DT outperforms its variants and strong meta offline RL baselines by a large margin with a trajectory prompt containing only a few timesteps. Prompt-DT is also robust to prompt length changes and can generalize to out-of-distribution (OOD) environments.

Key facts. Cited by 143 papers in the map (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 43 references that are in the map. First posted on arXiv 27 Jun 2022.

Builds on

36 of 43 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021Generalized Decision Transformer for Offline Hindsight Information Matching19 Nov
  2. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun
  4. 2020Offline Meta-Reinforcement Learning with Advantage Weighting13 Aug

Other references · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2016Matching Networks for One Shot Learningcited by 2,246
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  9. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  10. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  11. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
20 more
  1. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  2. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  3. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  4. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  5. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  6. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  7. 2019Meta-Learning with Implicit Gradientscited by 523
  8. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  9. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  10. 2017One-Shot Imitation Learningcited by 232
  11. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  12. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  13. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
  14. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  15. 2020BADGR: An Autonomous Self-Supervised Learning-Based Navigation Systemcited by 97
  16. 2016Learning feed-forward one-shot learnerscited by 93
  17. 2016Low Data Drug Discovery with One-shot Learningcited by 91
  18. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  19. 2018TACO: Learning Task Decomposition via Temporal Alignment for Controlcited by 61
  20. 2019Off-Policy Policy Gradient Algorithms by Constraining the State Distribution Shiftcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 34; 2024: 51; 2025: 29; 2026: 17.

143 papers.

Sort:

By area:

  1. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  2. 2023Large Language Models as General Pattern Machinescited by 196
  3. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 186
  4. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
  5. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  6. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  7. 2024Vision Language Models are In-Context Value Learnerscited by 84
  8. 2024Can large language models explore in-context?cited by 75
  9. 2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 69
  10. 2023Elastic Decision Transformercited by 66
  11. 2023Constrained Decision Transformer for Offline Safe Reinforcement Learningcited by 65
  12. 2023MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RLcited by 65
38 more
  1. 2024In-Context Imitation Learning via Next-Token Predictioncited by 65
  2. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  3. 2023TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Modelscited by 48
  4. 2023Generalization to New Sequential Decision Making Tasks with In-Context Learningcited by 39
  5. 2023What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?cited by 36
  6. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 36
  7. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  8. 2023Hyper-Decision Transformer for Efficient Online Policy Adaptationcited by 32
  9. 2024LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrationscited by 32
  10. 2022Large Language Models can Implement Policy Iterationcited by 31
  11. 2023Learning to Modulate pre-trained Models in RLcited by 28
  12. 2024LLMs Are In-Context Bandit Reinforcement Learnerscited by 27
  13. 2023Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Makingcited by 26
  14. 2023Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learningcited by 24
  15. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  16. 2023In-Context Reinforcement Learning for Variable Action Spacescited by 23
  17. 2023Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targetscited by 22
  18. 2023Zero-Shot Reinforcement Learning from Low Quality Datacited by 21
  19. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  20. 2023Prompt-Tuning Decision Transformer with Preference Rankingcited by 19
  21. 2023Large Sequence Models for Sequential Decision-Making: A Surveycited by 19
  22. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  23. 2024One-Shot Imitation Learning with Invariance Matching for Robotic Manipulationcited by 18
  24. 2024SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulationcited by 18
  25. 2025GUI-Bee: Align GUI Action Grounding to Novel Environments via Autonomous Explorationcited by 18
  26. 2023SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learningcited by 16
  27. 2024Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learningcited by 16
  28. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglementcited by 16
  29. 2022Transformers are Adaptable Task Plannerscited by 15
  30. 2023Skill Transformer: A Monolithic Policy for Mobile Manipulationcited by 15
  31. 2025Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformercited by 15
  32. 2023Rethinking Decision Transformer via Hierarchical Reinforcement Learningcited by 14
  33. 2024HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 14
  34. 2025Generative Models in Decision Making: A Surveycited by 14
  35. 2023Prompt Tuning of Deep Neural Networks for Speaker-adaptive Visual Speech Recognitioncited by 13
  36. 2024Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretrainingcited by 13
  37. 2024Return Augmented Decision Transformer for Off-Dynamics Reinforcement Learningcited by 13
  38. 2022A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcited by 12

50 most cited of 143 All 143 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM-Driven Policy Diffusion: Enhancing Generalization in Offline Reinforcement Learning
  2. 2026Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner
  3. 2025Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?
  4. 2026Adaptation of Generalist Robot Policies with Minimal Data
  5. 2025Adjusting the Output of Decision Transformer with Action Gradient
  6. 2023Multi-Objective Decision Transformers for Offline Reinforcement Learning
  7. 2026From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models
  8. 2025PromptLoop: Plug-and-Play Prompt Refinement via Latent Feedback for Diffusion Model Alignment