# AI Tax: The Hidden Cost of AI Data Center Applications

arXiv: https://arxiv.org/abs/2007.10571 · HTML: https://mlsanity.com/paper/2007.10571/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2020.

## Built on by (most cited first, 4 of 4)

- [CFU Playground: Full-Stack Open-Source Framework for Tiny Machine Learning (tinyML) Acceleration on FPGAs](https://mlsanity.com/paper/2201.01863/) (2022)
- [ULEEN: A Novel Architecture for Ultra Low-Energy Edge Neural Networks](https://mlsanity.com/paper/2304.10618/) (2023)
- [TaxBreak: Unmasking the Hidden Costs of LLM Inference Through Overhead Decomposition](https://mlsanity.com/paper/2603.12465/) (2026)
- [Beyond Inference: Performance Analysis of DNN Server Overheads for Computer Vision](https://mlsanity.com/paper/2403.12981/) (2024)

## Abstract (arXiv)

Artificial intelligence and machine learning are experiencing widespread adoption in industry and academia. This has been driven by rapid advances in the applications and accuracy of AI through increasingly complex algorithms and models; this, in turn, has spurred research into specialized hardware AI accelerators. Given the rapid pace of advances, it is easy to forget that they are often developed and evaluated in a vacuum without considering the full application environment. This paper emphasizes the need for a holistic, end-to-end analysis of AI workloads and reveals the "AI tax." We deploy and characterize Face Recognition in an edge data center. The application is an AI-centric edge video analytics application built using popular open source infrastructure and ML tools. Despite using state-of-the-art AI and ML algorithms, the application relies heavily on pre-and post-processing code. As AI-centric applications benefit from the acceleration promised by accelerators, we find they impose stresses on the hardware and software infrastructure: storage and network bandwidth become major bottlenecks with increasing AI acceleration. By specializing for AI applications, we show that a purpose-built edge data center can be designed for the stresses of accelerated AI at 15% lower TCO than one derived from homogeneous servers and infrastructure.