# PandasCorpus: A Resource of Real-World Pandas Workflows and Usage Patterns

arXiv: https://arxiv.org/abs/2608.14742 · HTML: https://mlsanity.com/paper/2608.14742/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 26 references that are in the map.
- First posted on arXiv 13 Aug 2026.

## Builds on (most cited first, 5 of 5)

- [Towards Scalable Dataframe Systems](https://mlsanity.com/paper/2001.00888/) (2020)
- [Better Code, Better Sharing:On the Need of Analyzing Jupyter Notebooks](https://mlsanity.com/paper/1906.05234/) (2019)
- [KGTorrent: A Dataset of Python Jupyter Notebooks from Kaggle](https://mlsanity.com/paper/2103.10558/) (2021)
- [Evaluation of Dataframe Libraries for Data Preparation on a Single Machine](https://mlsanity.com/paper/2312.11122/) (2023)
- [Xorbits: Automating Operator Tiling for Distributed Data Science](https://mlsanity.com/paper/2401.00865/) (2023)

## Abstract (arXiv)

Pandas has emerged as the de facto library for data processing and machine learning, widely used for tasks, such as data loading, transformation, and analysis. Despite its ubiquity, there has been limited systematic investigation into how Pandas is used in real-world projects and how typical workflows are composed in practice. To address this gap, we introduce PandasCorpus, a dataset curated from GitHub repositories that captures real-world Pandas workflows at scale. In this work, a workflow refers to Pandas-based code contained in Jupyter notebooks, a prevalent medium for writing, executing, and sharing data analysis code. The dataset comprises 139k notebooks from approximately 100k repositories and captures more than 4M Pandas API calls spanning 136 distinct operations. Beyond dataset construction, we characterize workflows using structural and Pandas-specific features and analyze notebook evolution between 2015 and 2025. Our study examines code executability, notebook size, and recurring sequences of Pandas operations, providing empirical insights into how Pandas is used in practice. The resulting corpus offers a reusable resource for studying data analysis workflows, Pandas usage patterns, and library-aware code composition. Both the dataset and the extraction pipeline are publicly available via GitHub and Zenodo.