mlsanityData to 7 Oct 2026

Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challenges

arXiv 2401.00031 ↗First posted 29 Dec 2023cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Decision-making is a dynamic process requiring perception, memory, and reasoning to make choices and find optimal policies. Traditional approaches to decision-making suffer from sample efficiency and generalization, while large-scale self-supervised pretraining has enabled fast adaptation with fine-tuning or few-shot learning in language and vision. We thus argue to integrate knowledge acquired from generic large-scale self-supervised pretraining into downstream decision-making problems. We propose Pretrain-Then-Adapt pipeline and survey recent work on data collection, pretraining objectives and adaptation strategies for decision-making pretraining and downstream inference. Finally, we identify critical challenges and future directions for developing decision foundation model with the help of generic and flexible self-supervised pretraining.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 29 Dec 2023.

Builds on

46 of 59 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  11. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  12. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
34 more
  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  4. 2018DeepMind Control Suitecited by 935
  5. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  6. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  7. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  8. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  9. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  10. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  11. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  12. 2022Multi-Game Decision Transformerscited by 212
  13. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  14. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
  15. 2018Optimization of Molecules via Deep Reinforcement Learningcited by 199
  16. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 186
  17. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  18. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  19. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 143
  20. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
  21. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  22. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 90
  23. 2019Learning Action Representations for Reinforcement Learningcited by 75
  24. 2018Deep Reinforcement Learning for Traffic Light Control in Vehicular Networkscited by 51
  25. 2022Masked Autoencoding for Scalable and Generalizable Decision Makingcited by 44
  26. 2023SMART: Self-supervised Multi-task pretrAining with contRol Transformerscited by 44
  27. 2023Masked Trajectory Models for Prediction, Representation, and Controlcited by 40
  28. 2023Future-conditioned Unsupervised Pretraining for Decision Transformercited by 26
  29. 2023Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targetscited by 22
  30. 2023Large Sequence Models for Sequential Decision-Making: A Surveycited by 19
  31. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  32. 2023Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learningcited by 10
  33. 2023RePreM: Representation Pre-training with Masked Model for Reinforcement Learningcited by 5
  34. 2023PASTA: Pretrained Action-State Transformer Agentscited by 5

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learningcited by 10
  2. 2024MoRE: Multi-Modal Contrastive Pre-training with Transformers on X-Rays, ECGs, and Diagnostic Reportcited by 3