# What Do Developers Ask About ML Libraries? A Large-scale Study Using Stack Overflow

arXiv: https://arxiv.org/abs/1906.11940 · HTML: https://mlsanity.com/paper/1906.11940/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jun 2019.

## Built on by (most cited first, 13 of 13)

- [Software Engineering for AI-Based Systems: A Survey](https://mlsanity.com/paper/2105.01984/) (2021)
- [Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process](https://mlsanity.com/paper/2110.10234/) (2021)
- [What are the Machine Learning best practices reported by practitioners on Stack Exchange?](https://mlsanity.com/paper/2301.10516/) (2023)
- [Machine Learning Application Development: Practitioners' Insights](https://mlsanity.com/paper/2112.15277/) (2021)
- [Bridging the Language Gap: An Empirical Study of Bindings for Open Source Machine Learning Libraries Across Software Package Ecosystems](https://mlsanity.com/paper/2201.07201/) (2022)
- [Challenges in Migrating Imperative Deep Learning Programs to Graph Execution: An Empirical Study](https://mlsanity.com/paper/2201.09953/) (2022)
- [Studying the Impact of TensorFlow and PyTorch Bindings on Machine Learning Software Quality](https://mlsanity.com/paper/2407.05466/) (2024)
- [An Empirical Study of OpenAI API Discussions on Stack Overflow](https://mlsanity.com/paper/2505.04084/) (2025)
- [Do Not Take It for Granted: Comparing Open-Source Libraries for Software Development Effort Estimation](https://mlsanity.com/paper/2207.01705/) (2022)
- [What Causes Exceptions in Machine Learning Applications? Mining Machine Learning-Related Stack Traces on Stack Overflow](https://mlsanity.com/paper/2304.12857/) (2023)
- [Towards machine learning guided by best practices](https://mlsanity.com/paper/2305.00233/) (2023)
- [Perspective of Software Engineering Researchers on Machine Learning Practices Regarding Research, Review, and Education](https://mlsanity.com/paper/2411.19304/) (2024)
- [Speculative Automated Refactoring of Imperative Deep Learning Programs to Graph Execution](https://mlsanity.com/paper/2504.05424/) (2025)

## Abstract (arXiv)

Modern software systems are increasingly including machine learning (ML) as an integral component. However, we do not yet understand the difficulties faced by software developers when learning about ML libraries and using them within their systems. To that end, this work reports on a detailed (manual) examination of 3,243 highly-rated Q&A posts related to ten ML libraries, namely Tensorflow, Keras, scikit-learn, Weka, Caffe, Theano, MLlib, Torch, Mahout, and H2O, on Stack Overflow, a popular online technical Q&A forum. We classify these questions into seven typical stages of an ML pipeline to understand the correlation between the library and the stage. Then we study the questions and perform statistical analysis to explore the answer to four research objectives (finding the most difficult stage, understanding the nature of problems, nature of libraries and studying whether the difficulties stayed consistent over time). Our findings reveal the urgent need for software engineering (SE) research in this area. Both static and dynamic analyses are mostly absent and badly needed to help developers find errors earlier. While there has been some early research on debugging, much more work is needed. API misuses are prevalent and API design improvements are sorely needed. Last and somewhat surprisingly, a tug of war between providing higher levels of abstractions and the need to understand the behavior of the trained model is prevalent.