mlsanityData to 7 Oct 2026

Foundation Models for Decision Making: Problems, Methods, and Opportunities

arXiv 2303.04129 ↗First posted 7 Mar 2023cs.AI cs.LGcited by 160

Not on the map. Why?

Cited by 36 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Foundation models pretrained on diverse data at scale have demonstrated extraordinary capabilities in a wide range of vision and language tasks. When such models are deployed in real world environments, they inevitably interface with other entities and agents. For example, language models are often used to interact with human beings through dialogue, and visual perception models are used to autonomously navigate neighborhood streets. In response to these developments, new paradigms are emerging for training foundation models to interact with other agents and perform long-term reasoning. These paradigms leverage the existence of ever-larger datasets curated for multimodal, multitask, and generalist interaction. Research at the intersection of foundation models and decision making holds tremendous promise for creating powerful new systems that can interact effectively across a diverse range of applications such as dialogue, autonomous driving, healthcare, education, and robotics. In this manuscript, we examine the scope of foundation models for decision making, and provide conceptual tools and technical background for understanding the problem space and exploring new research directions. We review recent approaches that ground foundation models in practical decision making applications through a variety of methods such as prompting, conditional generative modeling, planning, optimal control, and reinforcement learning, and discuss common challenges and open problems in the field.

Key facts. Cited by 160 papers in the map (data through 7 Oct 2026). Builds on 183 of its 230 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

183 of 230 references.

References · 183

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  7. 2022Training language models to follow instructions with human feedbackcited by 15,712
  8. 2017Proximal Policy Optimization Algorithmscited by 14,441
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  10. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  11. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  12. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
38 more
  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  12. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  13. 2015Continuous control with deep reinforcement learningcited by 3,279
  14. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  15. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  16. 2015Trust Region Policy Optimizationcited by 2,503
  17. 2022Emergent Abilities of Large Language Modelscited by 2,483
  18. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  19. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  20. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  21. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  22. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  23. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  24. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  25. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  26. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  27. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  28. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  29. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  30. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  31. 2016Generative Adversarial Imitation Learningcited by 1,576
  32. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  33. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  34. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  35. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  36. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  37. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  38. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218

50 most cited of 183 All 183 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 51; 2024: 51; 2025: 38; 2026: 20.

160 papers.

Sort:

By area:

  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  2. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  3. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  4. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  5. 2023Cognitive Architectures for Language Agentscited by 422
  6. 2024Understanding the planning of LLM agents: A surveycited by 414
  7. 2023Language Models can Solve Computer Taskscited by 371
  8. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  9. 2023Tool Learning with Foundation Modelscited by 302
  10. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 250
  11. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  12. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
38 more
  1. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  2. 2023Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Controlcited by 180
  3. 2023Video Language Planningcited by 160
  4. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  5. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  6. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
  7. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
  8. 2023On the Tool Manipulation Capability of Open-source Large Language Modelscited by 122
  9. 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 112
  10. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  11. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  12. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debatecited by 92
  13. 2023Compositional Foundation Models for Hierarchical Planningcited by 92
  14. 2025TesserAct: Learning 4D Embodied World Modelscited by 90
  15. 2024Video as the New Language for Real-World Decision Makingcited by 86
  16. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 82
  17. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 77
  18. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 77
  19. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 66
  20. 2023Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Modelscited by 66
  21. 2023Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorldcited by 65
  22. 2023Vision-Language Models as a Source of Rewardscited by 65
  23. 2024CFBench: A Comprehensive Constraints-Following Benchmark for LLMscited by 65
  24. 2024AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policiescited by 50
  25. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  26. 2023Tree-Planner: Efficient Close-loop Task Planning with Large Language Modelscited by 48
  27. 2024COMBO: Compositional World Models for Embodied Multi-Agent Cooperationcited by 47
  28. 2024RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learningcited by 45
  29. 2023Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergencecited by 42
  30. 2023LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzlescited by 41
  31. 2023Statler: State-Maintaining Language Models for Embodied Reasoningcited by 40
  32. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  33. 2023CoPAL: Corrective Planning of Robot Actions with Large Language Modelscited by 39
  34. 2024Learning to Ask: When LLM Agents Meet Unclear Instructioncited by 38
  35. 2023Hierarchical Prompting Assists Large Language Model on Web Navigationcited by 36
  36. 2023Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizercited by 35
  37. 2024ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarioscited by 35
  38. 2024HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environmentscited by 32

50 most cited of 160 All 160 (JSON)