# Evidence-Driven Decision Support for AI Model Selection in Research Software Engineering

arXiv: https://arxiv.org/abs/2512.11984 · HTML: https://mlsanity.com/paper/2512.11984/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 82 references that are in the map.
- First posted on arXiv 12 Dec 2025.

## Builds on (most cited first, 26 of 26)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [A Survey of the Recent Architectures of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1901.06032/) (2019)
- [Deep Learning Based Text Classification: A Comprehensive Review](https://mlsanity.com/paper/2004.03705/) (2020)
- [Does Prompt Formatting Have Any Impact on LLM Performance?](https://mlsanity.com/paper/2411.10541/) (2024)
- [Evaluating the Zero-shot Robustness of Instruction-tuned Language Models](https://mlsanity.com/paper/2306.11270/) (2023)
- [We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs](https://mlsanity.com/paper/2406.10279/) (2024)
- [An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry](https://mlsanity.com/paper/2303.02552/) (2023)
- [Software Engineering Challenges of Deep Learning](https://mlsanity.com/paper/1810.12034/) (2018)
- [Model Selection Techniques -- An Overview](https://mlsanity.com/paper/1810.09583/) (2018)
- [Predicting the Future of AI with AI: High-quality link prediction in an exponentially growing knowledge network](https://mlsanity.com/paper/2210.00881/) (2022)
- [Open (Clinical) LLMs are Sensitive to Instruction Phrasings](https://mlsanity.com/paper/2407.09429/) (2024)
- [Bias and Fairness in Chatbots: An Overview](https://mlsanity.com/paper/2309.08836/) (2023)
- [Fast and Informative Model Selection using Learning Curve Cross-Validation](https://mlsanity.com/paper/2111.13914/) (2021)
- [The Pace of Artificial Intelligence Innovations: Speed, Talent, and Trial-and-Error](https://mlsanity.com/paper/2009.01812/) (2020)
- [When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs](https://mlsanity.com/paper/2508.11383/) (2025)
- [DFSeer: A Visual Analytics Approach to Facilitate Model Selection for Demand Forecasting](https://mlsanity.com/paper/2005.03244/) (2020)
- [Artificial intelligence adoption in the physical sciences, natural sciences, life sciences, social sciences and the arts and humanities: A bibliometric analysis of research publications from 1960-2021](https://mlsanity.com/paper/2306.09145/) (2023)
- [Capturing Software Architecture Knowledge for Pattern-Driven Design](https://mlsanity.com/paper/2005.08393/) (2020)
- [Understanding User Intent Modeling for Conversational Recommender Systems: A Systematic Literature Review](https://mlsanity.com/paper/2308.08496/) (2023)
- [A Review of Fairness and A Practical Guide to Selecting Context-Appropriate Fairness Metrics in Machine Learning](https://mlsanity.com/paper/2411.06624/) (2024)
- [Auto-Model: Utilizing Research Papers and HPO Techniques to Deal with the CASH problem](https://mlsanity.com/paper/1910.10902/) (2019)
- [A Decision Model for Decentralized Autonomous Organization Platform Selection: Three Industry Case Studies](https://mlsanity.com/paper/2107.14093/) (2021)
- [An Empirical Study on Decision-Making Aspects in Responsible Software Engineering for AI](https://mlsanity.com/paper/2501.15691/) (2025)
- [Empirical Evaluation of AI-Assisted Software Package Selection: A Knowledge Graph Approach](https://mlsanity.com/paper/2508.05693/) (2025)

## Built on by (most cited first, 1 of 1)

- [HugSelect: An Explainable Multi-Criteria Decision-Support Framework for foundation-model selection](https://mlsanity.com/paper/2608.08069/) (2026)

## Abstract (arXiv)

The rapid proliferation of artificial intelligence (AI) models and methods presents growing challenges for research software engineers and researchers who must select, integrate, and maintain appropriate models within complex research workflows. Model selection is often performed in an ad hoc manner, relying on fragmented metadata and individual expertise, which can undermine reproducibility, transparency, and overall research software quality. This work proposes a structured and evidence-driven approach to support AI model selection that aligns with both technical and contextual requirements. We conceptualize AI model selection as a Multi-Criteria Decision-Making (MCDM) problem and introduce an evidence-based decision-support framework that integrates automated data collection pipelines, a structured knowledge graph, and MCDM principles. Following the Design Science Research methodology, the proposed framework (ModelSelect) is empirically validated through 50 real-world case studies and comparative experiments against leading generative AI systems. The evaluation results show that ModelSelect produces reliable, interpretable, and reproducible recommendations that closely align with expert reasoning. Across the case studies, the framework achieved high coverage and strong rationale alignment in both model and library recommendation tasks, performing comparably to generative AI assistants while offering superior traceability and consistency. By framing AI model selection as an MCDM problem, this work establishes a rigorous foundation for transparent and reproducible decision support in research software engineering. The proposed framework provides a scalable and explainable pathway for integrating empirical evidence into AI model recommendation processes, ultimately improving the quality and robustness of research software decision-making.