# Empirical Evaluation of AI-Assisted Software Package Selection: A Knowledge Graph Approach

arXiv: https://arxiv.org/abs/2508.05693 · HTML: https://mlsanity.com/paper/2508.05693/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 104 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Builds on (most cited first, 14 of 14)

- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs](https://mlsanity.com/paper/2406.10279/) (2024)
- [An Empirical Comparison of Dependency Network Evolution in Seven Software Packaging Ecosystems](https://mlsanity.com/paper/1710.04936/) (2017)
- [Transforming Software Development: Evaluating the Efficiency and Challenges of GitHub Copilot in Real-World Projects](https://mlsanity.com/paper/2406.17910/) (2024)
- [Capturing Software Architecture Knowledge for Pattern-Driven Design](https://mlsanity.com/paper/2005.08393/) (2020)
- [An Empirical Analysis of the Python Package Index (PyPI)](https://mlsanity.com/paper/1907.11073/) (2019)
- [A Large-Scale Security-Oriented Static Analysis of Python Packages in PyPI](https://mlsanity.com/paper/2107.12699/) (2021)
- [Understanding User Intent Modeling for Conversational Recommender Systems: A Systematic Literature Review](https://mlsanity.com/paper/2308.08496/) (2023)
- [Open Source, Open Threats? Investigating Security Challenges in Open-Source Software](https://mlsanity.com/paper/2506.12995/) (2025)
- [On the Impact of Micro-Packages: An Empirical Study of the npm JavaScript Ecosystem](https://mlsanity.com/paper/1709.04638/) (2017)
- [A Decision Model for Decentralized Autonomous Organization Platform Selection: Three Industry Case Studies](https://mlsanity.com/paper/2107.14093/) (2021)
- [What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI](https://mlsanity.com/paper/2505.17418/) (2025)
- [Open Source Software: An Approach to Controlling Usage and Risk in Application Ecosystems](https://mlsanity.com/paper/2206.10358/) (2022)
- [An Empirical Study of Package Management Issues via Stack Overflow](https://mlsanity.com/paper/2211.11197/) (2022)

## Built on by (most cited first, 2 of 2)

- [Evidence-Driven Decision Support for AI Model Selection in Research Software Engineering](https://mlsanity.com/paper/2512.11984/) (2025)
- [HugSelect: An Explainable Multi-Criteria Decision-Support Framework for foundation-model selection](https://mlsanity.com/paper/2608.08069/) (2026)

## Abstract (arXiv)

Selecting third-party software packages in open-source ecosystems like Python is challenging due to the large number of alternatives and limited transparent evidence for comparison. Generative AI tools are increasingly used in development workflows, but their suggestions often overlook dependency evaluation, emphasize popularity over suitability, and lack reproducibility. This creates risks for projects that require transparency, long-term reliability, maintainability, and informed architectural decisions. This study formulates software package selection as a Multi-Criteria Decision-Making (MCDM) problem and proposes a data-driven framework for technology evaluation. Automated data pipelines continuously collect and integrate software metadata, usage trends, vulnerability information, and developer sentiment from GitHub, PyPI, and Stack Overflow. These data are structured into a decision model representing relationships among packages, domain features, and quality attributes. The framework is implemented in PySelect, a decision support system that uses large language models to interpret user intent and query the model to identify contextually appropriate packages. The approach is evaluated using 798,669 Python scripts from 16,887 GitHub repositories and a user study based on the Technology Acceptance Model. Results show high data extraction precision, improved recommendation quality over generative AI baselines, and positive user evaluations of usefulness and ease of use. This work introduces a scalable, interpretable, and reproducible framework that supports evidence-based software selection using MCDM principles, empirical data, and AI-assisted intent modeling.