mlsanityData to 14 Oct 2026

The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)

arXiv 2505.10640 ↗First posted 15 May 2025cs.SE cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Foundation Models (FMs) such as Large Language Models (LLMs) are reshaping the software industry by enabling FMware, systems that integrate these FMs as core components. In this KDD 2025 tutorial, we present a comprehensive exploration of FMware that combines a curated catalogue of challenges with real-world production concerns. We first discuss the state of research and practice in building FMware. We further examine the difficulties in selecting suitable models, aligning high-quality domain-specific data, engineering robust prompts, and orchestrating autonomous agents. We then address the complex journey from impressive demos to production-ready systems by outlining issues in system testing, optimization, deployment, and integration with legacy software. Drawing on our industrial experience and recent research in the area, we provide actionable insights and a technology roadmap for overcoming these challenges. Attendees will gain practical strategies to enable the creation of trustworthy FMware in the evolving technology landscape.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 101 references that are in the map. First posted on arXiv 15 May 2025.

Builds on

45 of 101 references.

References · 45

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  6. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  7. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  8. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  9. 2024A Survey on Large Language Models for Code Generationcited by 658
  10. 2024Understanding the planning of LLM agents: A surveycited by 421
  11. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  12. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
33 more
  1. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  2. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
  3. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  4. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
  5. 2023How is ChatGPT's behavior changing over time?cited by 270
  6. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  7. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 144
  8. 2023Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4cited by 125
  9. 2023Agents: An Open-source Framework for Autonomous Language Agentscited by 116
  10. 2024Best Practices and Lessons Learned on Synthetic Datacited by 106
  11. 2024CodeR: Issue Resolving with Multi-Agent and Task Graphscited by 91
  12. 2024Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrieverscited by 84
  13. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 82
  14. 2023The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AIcited by 73
  15. 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
  16. 2023A Survey on Large Language Models for Software Engineeringcited by 59
  17. 2024Evaluating LLMs at Detecting Errors in LLM Responsescited by 50
  18. 2024Understanding the Performance and Estimating the Cost of LLM Fine-Tuningcited by 48
  19. 2020Behavioral Use Licensing for Responsible AIcited by 46
  20. 2024Current state of LLM Risks and AI Guardrailscited by 46
  21. 2024Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmapcited by 30
  22. 2024Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arenacited by 26
  23. 2023(Why) Is My Prompt Getting Worse? Rethinking Regression Testing for Evolving LLM APIscited by 23
  24. 2023KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graphcited by 22
  25. 2023Data Selection for Fine-tuning Large Language Models Using Transferred Shapley Valuescited by 21
  26. 2024An Empirical Study on Challenges for LLM Application Developerscited by 18
  27. 2023Building Your Own Product Copilot: Challenges, Opportunities, and Needscited by 17
  28. 2024LLM-Pilot: Characterize and Optimize Performance of your LLM Inference Servicescited by 14
  29. 2024Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Productscited by 11
  30. 2021DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testingcited by 8
  31. 2023Model Reporting for Certifiable AI: A Proposal from Merging EU Regulation into AI Developmentcited by 6
  32. 2024On the Workflows and Smells of Leaderboard Operations (LBOps): An Exploratory Study of Foundation Model Leaderboardscited by 6
  33. 2013Architecture and Production Readiness Reviews in Practicecited by 2

Built on by

Not cited yet.