Software Engineering and Foundation Models: Insights from Industry Blogs Using a Jury of Foundation Models
Not on the map. Why?
Abstract (arXiv)
Foundation models (FMs) such as large language models (LLMs) have significantly impacted many fields, including software engineering (SE). The interaction between SE and FMs has led to the integration of FMs into SE practices (FM4SE) and the application of SE methodologies to FMs (SE4FM). While several literature surveys exist on academic contributions to these trends, we are the first to provide a practitioner's view. We analyze 155 FM4SE and 997 SE4FM blog posts from leading technology companies, leveraging an FM-powered surveying approach to systematically label and summarize the discussed activities and tasks. We observed that while code generation is the most prominent FM4SE task, FMs are leveraged for many other SE activities such as code understanding, summarization, and API recommendation. The majority of blog posts on SE4FM are about model deployment & operation, and system architecture & orchestration. Although the emphasis is on cloud deployments, there is a growing interest in compressing FMs and deploying them on smaller devices such as edge or mobile devices. We outline eight future research directions inspired by our gained insights, aiming to bridge the gap between academic findings and real-world applications. Our study not only enriches the body of knowledge on practical applications of FM4SE and SE4FM but also demonstrates the utility of FMs as a powerful and efficient approach in conducting literature surveys within technical and grey literature domains. Our dataset, results, code and used prompts can be found in our online replication package at https://github.com/SAILResearch/fmse-blogs.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 11 Oct 2024.
Builds on
33 of 51 references.
References · 33
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
- 2024Qwen2 Technical Reportcited by 2,076
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
21 more
- 2023A Survey on Evaluation of Large Language Modelscited by 1,227
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
- 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
- 2023Large Language Modelscited by 774
- 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
- 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
- 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
- 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
- 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
- 2023Recognize Anything: A Strong Image Tagging Modelcited by 305
- 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
- 2023A Survey on Model Compression for Large Language Modelscited by 263
- 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
- 2023Large Language Models for Software Engineering: Survey and Open Problemscited by 178
- 2023Software Testing with Large Language Models: Survey, Landscape, and Visioncited by 163
- 2023A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challengescited by 75
- 2021Software Engineering for AI-Based Systems: A Surveycited by 68
- 2024Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmapcited by 30
- 2023Bug Characterization in Machine Learning-based Systemscited by 10
Built on by
Citing papers per year: 2024: 1; 2025: 2; 2026: 3.
6 papers.
- 2025Agentic Software Engineering: Foundational Pillars and a Research Roadmapcited by 40
- 2024The Impact of Prompt Programming on Function-Level Code Generationcited by 13
- 2026Do AI Coding Agents Log Like Humans? An Empirical Studycited by 2
- 2025Search-Based Software Engineering and AI Foundation Models: Current Landscape and Future Roadmapcited by 0
- 2026Beyond the Autoregressive Horizon: A Comprehensive Survey of Diffusion Models, World Modelling, and State Space Models for Codecited by 0
- 2026Automatic Model Card Generation Using an LLMcited by 0
- 2026Automatic Model Card Generation Using an LLM25 Aug 2026
- 2026Do AI Coding Agents Log Like Humans? An Empirical Study10 Apr 2026
- 2026Beyond the Autoregressive Horizon: A Comprehensive Survey of Diffusion Models, World Modelling, and State Space Models for Code9 Apr 2026
- 2025Agentic Software Engineering: Foundational Pillars and a Research Roadmap7 Sep 2025
- 2025Search-Based Software Engineering and AI Foundation Models: Current Landscape and Future Roadmap26 May 2025
- 2024The Impact of Prompt Programming on Function-Level Code Generation29 Dec 2024