mlsanityData to 7 Oct 2026

Software Engineering and Foundation Models: Insights from Industry Blogs Using a Jury of Foundation Models

arXiv 2410.09012 ↗First posted 11 Oct 2024cs.SE cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Foundation models (FMs) such as large language models (LLMs) have significantly impacted many fields, including software engineering (SE). The interaction between SE and FMs has led to the integration of FMs into SE practices (FM4SE) and the application of SE methodologies to FMs (SE4FM). While several literature surveys exist on academic contributions to these trends, we are the first to provide a practitioner's view. We analyze 155 FM4SE and 997 SE4FM blog posts from leading technology companies, leveraging an FM-powered surveying approach to systematically label and summarize the discussed activities and tasks. We observed that while code generation is the most prominent FM4SE task, FMs are leveraged for many other SE activities such as code understanding, summarization, and API recommendation. The majority of blog posts on SE4FM are about model deployment & operation, and system architecture & orchestration. Although the emphasis is on cloud deployments, there is a growing interest in compressing FMs and deploying them on smaller devices such as edge or mobile devices. We outline eight future research directions inspired by our gained insights, aiming to bridge the gap between academic findings and real-world applications. Our study not only enriches the body of knowledge on practical applications of FM4SE and SE4FM but also demonstrates the utility of FMs as a powerful and efficient approach in conducting literature surveys within technical and grey literature domains. Our dataset, results, code and used prompts can be found in our online replication package at https://github.com/SAILResearch/fmse-blogs.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 11 Oct 2024.

Builds on

33 of 51 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  9. 2022Emergent Abilities of Large Language Modelscited by 2,483
  10. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  11. 2024Qwen2 Technical Reportcited by 2,076
  12. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
21 more
  1. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  3. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  4. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  5. 2023Large Language Modelscited by 774
  6. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  7. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  8. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  9. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  10. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  11. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  12. 2023Recognize Anything: A Strong Image Tagging Modelcited by 305
  13. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
  14. 2023A Survey on Model Compression for Large Language Modelscited by 263
  15. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
  16. 2023Large Language Models for Software Engineering: Survey and Open Problemscited by 178
  17. 2023Software Testing with Large Language Models: Survey, Landscape, and Visioncited by 163
  18. 2023A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challengescited by 75
  19. 2021Software Engineering for AI-Based Systems: A Surveycited by 68
  20. 2024Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmapcited by 30
  21. 2023Bug Characterization in Machine Learning-based Systemscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 3.

6 papers.