mlsanityData to 7 Oct 2026

A Case for Business Process-Specific Foundation Models

arXiv 2210.14739 ↗First posted 26 Oct 2022cs.AI cs.NEcited by 5

Not on the map. Why?

Abstract (arXiv)

The inception of large language models has helped advance state-of-the-art performance on numerous natural language tasks. This has also opened the door for the development of foundation models for other domains and data modalities such as images, code, and music. In this paper, we argue that business process data representations have unique characteristics that warrant the development of a new class of foundation models to handle tasks like process mining, optimization, and decision making. These models should also tackle the unique challenges of applying AI to business processes which include data scarcity, multi-modal representations, domain specific terminology, and privacy concerns.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 27 of its 53 references that are in the map. First posted on arXiv 26 Oct 2022.

Builds on

27 of 53 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2020Extracting Training Data from Large Language Modelscited by 2,144
  9. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  10. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  12. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
15 more
  1. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  2. 2019Publicly Available Clinical BERT Embeddingscited by 995
  3. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  4. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  5. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 424
  6. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
  7. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  8. 2016Predictive Business Process Monitoring with LSTM Neural Networkscited by 103
  9. 2017LSTM Networks for Data-Aware Remaining Time Prediction of Business Process Instancescited by 19
  10. 2020Detecting sudden and gradual drifts in business processes from execution tracescited by 8
  11. 2020A Conversational Digital Assistant for Intelligent Process Automationcited by 8
  12. 2020Conformance checking: A state-of-the-art literature reviewcited by 7
  13. 2021Risks of AI Foundation Models in Educationcited by 6
  14. 2020D3BA: A Tool for Optimizing Business Processes Using Non-Deterministic Planningcited by 4
  15. 2021Deep Reinforcement Learning for Resource Allocation in Business Processescited by 4

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  2. 2024Automating the Enterprise with Foundation Modelscited by 14
  3. 2023Synthesis of Mathematical programs from Natural Language Specificationscited by 8
  4. 2024How well can a large language model explain business processes as perceived by users?cited by 8
  5. 2025From Source to Target: Leveraging Transfer Learning for Predictive Process Monitoring in Organizationscited by 1