mlsanityData to 7 Oct 2026

Open-Source AI-based SE Tools: Opportunities and Challenges of Collaborative Software Learning

arXiv 2404.06201 ↗First posted 9 Apr 2024cs.SE cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have become instrumental in advancing software engineering (SE) tasks, showcasing their efficacy in code understanding and beyond. Like traditional SE tools, open-source collaboration is key in realising the excellent products. However, with AI models, the essential need is in data. The collaboration of these AI-based SE models hinges on maximising the sources of high-quality data. However, data especially of high quality, often holds commercial or sensitive value, making it less accessible for open-source AI-based SE projects. This reality presents a significant barrier to the development and enhancement of AI-based SE tools within the software engineering community. Therefore, researchers need to find solutions for enabling open-source AI-based SE models to tap into resources by different organisations. Addressing this challenge, our position paper investigates one solution to facilitate access to diverse organizational resources for open-source AI models, ensuring privacy and commercial sensitivities are respected. We introduce a governance framework centered on federated learning (FL), designed to foster the joint development and maintenance of open-source AI code models while safeguarding data privacy and security. Additionally, we present guidelines for developers on AI-based SE tool collaboration, covering data requirements, model architecture, updating strategies, and version control. Given the significant influence of data characteristics on FL, our research examines the effect of code data heterogeneity on FL performance.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 43 of its 90 references that are in the map. First posted on arXiv 9 Apr 2024.

Builds on

43 of 90 references.

References · 43

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  8. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  9. 2020Knowledge Distillation: A Surveycited by 1,578
  10. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  11. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  12. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
31 more
  1. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  2. 2023StarCoder: may the source be with you!cited by 790
  3. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  4. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  5. 2019Large Scale Incremental Learningcited by 727
  6. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
  7. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  8. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  9. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  10. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  11. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  12. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  13. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 266
  14. 2019Tackling Climate Change with Machine Learningcited by 264
  15. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  16. 2021Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Littlecited by 176
  17. 2023Software Testing with Large Language Models: Survey, Landscape, and Visioncited by 163
  18. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  19. 2021CURE: Code-Aware Neural Machine Translation for Automatic Program Repaircited by 85
  20. 2020A Blockchain-based Decentralized Federated Learning Framework with Committee Consensuscited by 83
  21. 2019Towards Efficient Model Compression via Learned Global Rankingcited by 75
  22. 2022A Broad Study of Pre-training for Domain Generalization and Adaptationcited by 64
  23. 2019Differential Privacy-enabled Federated Learning for Sensitive Health Datacited by 44
  24. 2020Dynamic Clustering in Federated Learningcited by 27
  25. 2023How Effective Are They? Exploring Large Language Model Based Fuzz Driver Generationcited by 21
  26. 2023Exploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMscited by 19
  27. 2017Agile Software Development Methods: Review and Analysiscited by 18
  28. 2021On the validity of pre-trained transformers for natural language processing in the software engineering domaincited by 11
  29. 2022Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacitiescited by 9
  30. 2022CommitBART: A Large Pre-trained Model for GitHub Commitscited by 6
  31. 2023FedCSD: A Federated Learning Based Approach for Code-Smell Detectioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Cross-Cloud Data Privacy Protection: Optimizing Collaborative Mechanisms of AI Systems by Integrating Federated Learning and LLMscited by 9
  2. 2026Artificial Intelligence in Open Source Software Engineering: A Foundation for Sustainabilitycited by 1
  3. 2025Towards Privacy-Preserving Code Generation: Differentially Private Code Language Modelscited by 0
  4. 2026From Conversation to Contribution: Characterizing Coding Agent in Open-Source Softwarecited by 0