# An Empirical Study of Challenges in Machine Learning Asset Management

arXiv: https://arxiv.org/abs/2402.15990 · HTML: https://mlsanity.com/paper/2402.15990/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 56 references that are in the map.
- First posted on arXiv 25 Feb 2024.

## Builds on (most cited first, 13 of 13)

- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [Is GPT-4 a Good Data Analyst?](https://mlsanity.com/paper/2305.15038/) (2023)
- [A Software Engineering Perspective on Engineering Machine Learning Systems: State of the Art and Challenges](https://mlsanity.com/paper/2012.07919/) (2020)
- [What's in a GitHub Star? Understanding Repository Starring Practices in a Social Coding Platform](https://mlsanity.com/paper/1811.07643/) (2018)
- [Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities](https://mlsanity.com/paper/2103.16007/) (2021)
- [Management of Machine Learning Lifecycle Artifacts: A Survey](https://mlsanity.com/paper/2210.11831/) (2022)
- [Why Modern Open Source Projects Fail](https://mlsanity.com/paper/1707.02327/) (2017)
- [Scanflow: A multi-graph framework for Machine Learning workflow management, supervision, and debugging](https://mlsanity.com/paper/2111.03003/) (2021)

## Built on by (most cited first, 4 of 4)

- [A Large-Scale Study of Model Integration in ML-Enabled Software Systems](https://mlsanity.com/paper/2408.06226/) (2024)
- [Cataloguing Hugging Face Models to Software Engineering Activities: Automation and Findings](https://mlsanity.com/paper/2506.03013/) (2025)
- [On the Adoption of AI Coding Agents in Open-source Android and iOS Development](https://mlsanity.com/paper/2602.12144/) (2026)
- [A Two-Dimensional Study of the Model Context Protocol: Publication and Adoption](https://mlsanity.com/paper/2609.14721/) (2026)

## Abstract (arXiv)

In machine learning (ML), efficient asset management, including ML models, datasets, algorithms, and tools, is vital for resource optimization, consistent performance, and a streamlined development lifecycle. This enables quicker iterations, adaptability, reduced development-to-deployment time, and reliable outputs. Despite existing research, a significant knowledge gap remains in operational challenges like model versioning, data traceability, and collaboration, which are crucial for the success of ML projects. Our study aims to address this gap by analyzing 15,065 posts from developer forums and platforms, employing a mixed-method approach to classify inquiries, extract challenges using BERTopic, and identify solutions through open card sorting and BERTopic clustering. We uncover 133 topics related to asset management challenges, grouped into 16 macro-topics, with software dependency, model deployment, and model training being the most discussed. We also find 79 solution topics, categorized under 18 macro-topics, highlighting software dependency, feature development, and file management as key solutions. This research underscores the need for further exploration of identified pain points and the importance of collaborative efforts across academia, industry, and the research community.