# Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures

arXiv: https://arxiv.org/abs/2601.03988 · HTML: https://mlsanity.com/paper/2601.03988/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 69 references that are in the map.
- First posted on arXiv 7 Jan 2026.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning](https://mlsanity.com/paper/2002.05651/) (2020)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [The Prompt Report: A Systematic Survey of Prompt Engineering Techniques](https://mlsanity.com/paper/2406.06608/) (2024)
- [A Comprehensive Study on Deep Learning Bug Characteristics](https://mlsanity.com/paper/1906.01388/) (2019)
- ["We Need Structured Output": Towards User-centered Constraints on Large Language Model Output](https://mlsanity.com/paper/2404.07362/) (2024)
- [Characterizing Technical Debt and Antipatterns in AI-Based Systems: A Systematic Mapping Study](https://mlsanity.com/paper/2103.09783/) (2021)
- [Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs](https://mlsanity.com/paper/2401.10065/) (2024)
- [NLPerturbator: Studying the Robustness of Code LLMs to Natural Language Variations](https://mlsanity.com/paper/2406.19783/) (2024)
- [Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks](https://mlsanity.com/paper/2102.12592/) (2021)
- [The Art and Practice of Data Science Pipelines: A Comprehensive Study of Data Science Pipelines In Theory, In-The-Small, and In-The-Large](https://mlsanity.com/paper/2112.01590/) (2021)
- [Towards Evaluation Guidelines for Empirical Studies involving LLMs](https://mlsanity.com/paper/2411.07668/) (2024)
- [How can AI Automate End-to-End Data Science?](https://mlsanity.com/paper/1910.14436/) (2019)
- [Why do Machine Learning Notebooks Crash? An Empirical Study on Public Python Jupyter Notebooks](https://mlsanity.com/paper/2411.16795/) (2024)
- [CORAL: COde RepresentAtion Learning with Weakly-Supervised Transformers for Analyzing Data Analysis](https://mlsanity.com/paper/2008.12828/) (2020)
- [Static Analysis Driven Enhancements for Comprehension in Machine Learning Notebooks](https://mlsanity.com/paper/2301.04419/) (2023)
- [A Flexible Cell Classification for ML Projects in Jupyter Notebooks](https://mlsanity.com/paper/2403.07562/) (2024)
- [Reverse Engineering User Stories from Code using Large Language Models](https://mlsanity.com/paper/2509.19587/) (2025)
- [REx86: A Local Large Language Model for Assisting in x86 Assembly Reverse Engineering](https://mlsanity.com/paper/2510.20975/) (2025)

## Built on by (most cited first, 1 of 1)

- [Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures](https://mlsanity.com/paper/2610.10261/) (2026)

## Abstract (arXiv)

Background: Extracting the stages that structure Machine Learning (ML) pipelines from source code is key for gaining a deeper understanding of data science practices. However, the diversity caused by the constant evolution of the ML ecosystem (e.g., algorithms, libraries, datasets) makes this task challenging. Existing approaches either depend on non-scalable, manual labeling, or on ML classifiers that do not properly support the diversity of the domain. These limitations highlight the need for more flexible and reliable solutions. Objective: We evaluate whether Small Language Models (SLMs) can leverage their code understanding and classification abilities to address these limitations, and subsequently how they can advance our understanding of data science practices. Method: We conduct a confirmatory study based on two reference works selected for their relevance regarding current state-of-the-art's limitations. First, we compare several SLMs using Cochran's Q test. The best-performing model is then evaluated against the reference studies using two distinct McNemar's tests. We further analyze how variations in taxonomy definitions affect performance through an additional Cochran's Q test. Finally, a goodness-of-fit analysis is conducted using Pearson's chi-squared tests to compare our insights on data science practices with those from prior studies.