mlsanityData to 7 Oct 2026

The Case for Developing a Foundation Model for Planning-like Tasks from Scratch

arXiv 2404.04540 ↗First posted 6 Apr 2024cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Foundation Models (FMs) have revolutionized many areas of computing, including Automated Planning and Scheduling (APS). For example, a recent study found them useful for planning problems: plan generation, language translation, model construction, multi-agent planning, interactive planning, heuristics optimization, tool integration, and brain-inspired planning. Besides APS, there are many seemingly related tasks involving the generation of a series of actions with varying guarantees of their executability to achieve intended goals, which we collectively call planning-like (PL) tasks like business processes, programs, workflows, and guidelines, where researchers have considered using FMs. However, previous works have primarily focused on pre-trained, off-the-shelf FMs and optionally fine-tuned them. This paper discusses the need for a comprehensive FM for PL tasks from scratch and explores its design considerations. We argue that such an FM will open new and efficient avenues for PL problem-solving, just like LLMs are creating for APS.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 20 of its 42 references that are in the map. First posted on arXiv 6 Apr 2024.

Builds on

20 of 42 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  8. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  9. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  10. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  11. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
  12. 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 266
8 more
  1. 2023Can Large Language Models Really Improve by Self-critiquing Their Own Plans?cited by 107
  2. 2024Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrappingcited by 89
  3. 2017Ethical Challenges in Data-Driven Dialogue Systemscited by 86
  4. 2023On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)cited by 85
  5. 2024On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)cited by 40
  6. 2015A Survey of Available Corpora for Building Data-Driven Dialogue Systemscited by 33
  7. 2022Process Knowledge-Infused AI: Towards User-level Explainability, Interpretability, and Safetycited by 18
  8. 2024From Cloud to Edge: Rethinking Generative AI for Low-Resource Design Challengescited by 5

Built on by

Cited by year

Citing papers per year: 2024: 3.

3 papers.

  1. 2024Neurosymbolic AI for Enhancing Instructability in Generative AIcited by 3
  2. 2024PLANTS: A Novel Problem and Dataset for Summarization of Planning-Like (PL) Taskscited by 0
  3. 2024A Novel Approach to Balance Convenience and Nutrition in Meals With Long-Term Group Recommendations and Reasoning on Multimodal Recipes and its Implementation in BEACONcited by 0