# Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures

arXiv: https://arxiv.org/abs/2610.10261 · HTML: https://mlsanity.com/paper/2610.10261/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 61 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Builds on (most cited first, 8 of 8)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [The Prompt Report: A Systematic Survey of Prompt Engineering Techniques](https://mlsanity.com/paper/2406.06608/) (2024)
- [How can AI Automate End-to-End Data Science?](https://mlsanity.com/paper/1910.14436/) (2019)
- [A Flexible Cell Classification for ML Projects in Jupyter Notebooks](https://mlsanity.com/paper/2403.07562/) (2024)
- [Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures](https://mlsanity.com/paper/2601.03988/) (2026)

## Abstract (arXiv)

Context: Once defined a taxonomy of stages structuring Machine Learning (ML) pipelines (e.g. Data Preprocessing, Modeling...), extracting these stages from source code is key for better understanding ML practices. However, the diversity caused by the constant evolution of ML (e.g., algorithms, datasets) makes this task challenging. Existing approaches either rely on non-scalable manual labeling or on classifiers that do not properly support domain's diversity. These limitations call for more reliable solutions. Objective: We evaluate whether Small Language Models (SLMs) can leverage their code understanding and classification abilities to address these limitations, and enhance our understanding of practices in ML. Method: We conduct a confirmatory study based on two relevant reference works representing current limitations in the state-of-the-art. We first compare several SLMs using Cochran's Q test, then evaluate the best-performing model against reference studies via two McNemar's tests. An additional Cochran's Q test examines how taxonomy definition variations affect the SLM performance. Finally, goodness-of-fit tests compare ML practice insights from SLM classification with those from prior studies. Results: First, we found that the taxonomy wording significantly impacts classification performance. Second, the best performing SLM yielded good results, yet, without outperforming other classifiers. Third, the three classification methods led to significantly different insights, with varying effect sizes, when exploring practices of data scientists. Conclusions: Limitations of existing classification methods bias our understanding of ML practices. While current SLMs show promising results without prior fine-tuning, they still exhibit common limitations, in addition to inference high costs challenging their applicability in large-scale studies.