# Energy Consumption of Dataframe Libraries for End-to-End Deep Learning Pipelines:A Comparative Analysis

arXiv: https://arxiv.org/abs/2511.08644 · HTML: https://mlsanity.com/paper/2511.08644/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 20 references that are in the map.
- First posted on arXiv 10 Nov 2025.

## Builds on (most cited first, 4 of 4)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Contemporary Recommendation Systems on Big Data and Their Applications: A Survey](https://mlsanity.com/paper/2206.02631/) (2022)
- [Evaluation of Dataframe Libraries for Data Preparation on a Single Machine](https://mlsanity.com/paper/2312.11122/) (2023)

## Built on by (most cited first, 1 of 1)

- [Improving the Energy-Efficiency of the Code Generated by LLMs through Effective Prompting](https://mlsanity.com/paper/2610.02571/) (2026)

## Abstract (arXiv)

This paper presents a detailed comparative analysis of the performance of three major Python data manipulation libraries - Pandas, Polars, and Dask - specifically when embedded within complete deep learning (DL) training and inference pipelines. The research bridges a gap in existing literature by studying how these libraries interact with substantial GPU workloads during critical phases like data loading, preprocessing, and batch feeding. The authors measured key performance indicators including runtime, memory usage, disk usage, and energy consumption (both CPU and GPU) across various machine learning models and datasets.