# Exploration of TPUs for AI Applications

arXiv: https://arxiv.org/abs/2309.08918 · HTML: https://mlsanity.com/paper/2309.08918/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 16 Sep 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [A Survey of Model Compression and Acceleration for Deep Neural Networks](https://mlsanity.com/paper/1710.09282/) (2017)
- [MLPerf Inference Benchmark](https://mlsanity.com/paper/1911.02549/) (2019)
- [Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation](https://mlsanity.com/paper/2004.09602/) (2020)
- [Generative Adversarial Super-Resolution at the Edge with Knowledge Distillation](https://mlsanity.com/paper/2209.03355/) (2022)
- [Scientific Image Restoration Anywhere](https://mlsanity.com/paper/1911.05878/) (2019)
- [Facial Expression Recognition at the Edge: CPU vs GPU vs VPU vs TPU](https://mlsanity.com/paper/2305.15422/) (2023)
- [GPTPU: Accelerating Applications using Edge Tensor Processing Units](https://mlsanity.com/paper/2107.05473/) (2021)
- [Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge TPU](https://mlsanity.com/paper/2204.03296/) (2022)
- [A Study on the Use of Edge TPUs for Eye Fundus Image Segmentation](https://mlsanity.com/paper/2207.12770/) (2022)
- [TPU-MLIR: A Compiler For TPU Using MLIR](https://mlsanity.com/paper/2210.15016/) (2022)
- [Edge Devices Inference Performance Comparison](https://mlsanity.com/paper/2306.12093/) (2023)
- [EdgeDRNN: Enabling Low-latency Recurrent Neural Network Edge Inference](https://mlsanity.com/paper/1912.12193/) (2019)
- [Benchmarking GPU and TPU Performance with Graph Neural Networks](https://mlsanity.com/paper/2210.12247/) (2022)

## Abstract (arXiv)

Tensor Processing Units (TPUs) are specialized hardware accelerators for deep learning developed by Google. This paper aims to explore TPUs in cloud and edge computing focusing on its applications in AI. We provide an overview of TPUs, their general architecture, specifically their design in relation to neural networks, compilation techniques and supporting frameworks. Furthermore, we provide a comparative analysis of Cloud and Edge TPU performance against other counterpart chip architectures. Our results show that TPUs can provide significant performance improvements in both cloud and edge computing. Additionally, this paper underscores the imperative need for further research in optimization techniques for efficient deployment of AI architectures on the Edge TPU and benchmarking standards for a more robust comparative analysis in edge computing scenarios. The primary motivation behind this push for research is that efficient AI acceleration, facilitated by TPUs, can lead to substantial savings in terms of time, money, and environmental resources.