mlsanityData to 7 Oct 2026

TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings

arXiv 2304.01433 ↗First posted 4 Apr 2023cs.AR cs.AI cs.LGcited by 198

Not on the map. Why?

Cited by 20 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In response to innovations in machine learning (ML) models, production workloads changed radically and rapidly. TPU v4 is the fifth Google domain specific architecture (DSA) and its third supercomputer for such ML models. Optical circuit switches (OCSes) dynamically reconfigure its interconnect topology to improve scale, availability, utilization, modularity, deployment, security, power, and performance; users can pick a twisted 3D torus topology if desired. Much cheaper, lower power, and faster than Infiniband, OCSes and underlying optical components are <5% of system cost and <3% of system power. Each TPU v4 includes SparseCores, dataflow processors that accelerate models that rely on embeddings by 5x-7x yet use only 5% of die area and power. Deployed since 2020, TPU v4 outperforms TPU v3 by 2.1x and improves performance/Watt by 2.7x. The TPU v4 supercomputer is 4x larger at 4096 chips and thus ~10x faster overall, which along with OCS flexibility helps large language models. For similar sized systems, it is ~4.3x-4.5x faster than the Graphcore IPU Bow and is 1.2x-1.7x faster and uses 1.3x-1.9x less power than the Nvidia A100. TPU v4s inside the energy-optimized warehouse scale computers of Google Cloud use ~3x less energy and produce ~20x less CO2e than contemporary DSAs in a typical on-premise data center.

Key facts. Cited by 198 papers in the map (data through 7 Oct 2026). Builds on 18 of its 65 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

18 of 65 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  6. 2019Green AIcited by 620
  7. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  8. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 183
  9. 2021GSPMD: General and Scalable Parallelization for ML Computation Graphscited by 109
  10. 2021Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Modelscited by 78
  11. 2022On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Modelscited by 34
  12. 2022TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobscited by 30
6 more
  1. 2021Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Trainingcited by 28
  2. 2021Searching for Fast Model Families on Datacenter Acceleratorscited by 22
  3. 2022Learning to Collide: Recommendation System Model Compression with Learned Hash Functionscited by 5
  4. 2022Mission Apollo: Landing Optical Circuit Switching at Datacenter Scalecited by 4
  5. 2023FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Modelscited by 4
  6. 2020COUDER: Robust Topology Engineering for Optical Circuit Switched Data Center Networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 36; 2024: 56; 2025: 73; 2026: 33.

198 papers.

Sort:

By area:

  1. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  2. 2024Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Modelscited by 232
  3. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 206
  4. 2023Simple synthetic data reduces sycophancy in large language modelscited by 174
  5. 2023FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Leadcited by 170
  6. 2023GenCast: Diffusion-based ensemble forecasting for medium-range weathercited by 169
  7. 2023SoundStorm: Efficient Parallel Audio Generationcited by 111
  8. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 101
  9. 2023Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLMcited by 92
  10. 2024Human Alignment of Large Language Models through Online Preference Optimisationcited by 90
  11. 2023Symbol tuning improves in-context learning in language modelscited by 72
  12. 2024Multi-turn Reinforcement Learning from Preference Human Feedbackcited by 65
38 more
  1. 2024Offline Regularised Reinforcement Learning for Large Language Models Alignmentcited by 48
  2. 2023Training Chain-of-Thought via Latent-Variable Inferencecited by 46
  3. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compressioncited by 41
  4. 2024Lightweight Deep Learning for Resource-Constrained Environments: A Surveycited by 38
  5. 2025Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architecturescited by 38
  6. 2023LMDX: Language Model-based Document Information Extraction and Localizationcited by 37
  7. 2025Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wastefulcited by 36
  8. 2023FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Corescited by 32
  9. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  10. 2024Revisiting Reliability in Large-Scale Machine Learning Research Clusterscited by 27
  11. 2024CALRec: Contrastive Alignment of Generative LLMs for Sequential Recommendationcited by 23
  12. 2024Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantizationcited by 22
  13. 2025PIM Is All You Need: A CXL-Enabled GPU-Free System for Large Language Model Inferencecited by 22
  14. 2023A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platformscited by 21
  15. 2023Scaling Down to Scale Up: A Cost-Benefit Analysis of Replacing OpenAI's LLM with Open Source SLMs in Productioncited by 21
  16. 2024Bigger is not Always Better: Scaling Properties of Latent Diffusion Modelscited by 21
  17. 2024TransformerFAM: Feedback attention is working memorycited by 21
  18. 2024A Survey: Collaborative Hardware and Software Design in the Era of Large Language Modelscited by 21
  19. 2025Robust Multi-Objective Preference Alignment with Online DPOcited by 21
  20. 2025AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimizationcited by 21
  21. 2024Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMscited by 18
  22. 2024SambaNova SN40L: Scaling the AI Memory Wall with Dataflow and Composition of Expertscited by 18
  23. 2025Ditto: Accelerating Diffusion Model via Temporal Value Similaritycited by 18
  24. 2024FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher informationcited by 17
  25. 2023DORSal: Diffusion for Object-centric Representations of Scenes et alcited by 16
  26. 2024Allo: A Programming Model for Composable Accelerator Designcited by 16
  27. 2024Forecasting GPU Performance for Deep Learning Training and Inferencecited by 15
  28. 2024MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from Microwatts to Megawatts for Sustainable AIcited by 15
  29. 2024APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Servingcited by 15
  30. 2024Flex-PE: Flexible and SIMD Multi-Precision Processing Element for AI Workloadscited by 15
  31. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 15
  32. 2023Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systemscited by 14
  33. 2023MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systemscited by 14
  34. 2024ConSmax: Hardware-Friendly Alternative Softmax with Learnable Parameterscited by 14
  35. 2024ReCycle: Resilient Training of Large DNNs using Pipeline Adaptationcited by 14
  36. 2024Generative AI for the Optimization of Next-Generation Wireless Networks: Basics, State-of-the-Art, and Open Challengescited by 14
  37. 2023Rail-only: A Low-Cost High-Performance Network for Training LLMs with Trillion Parameterscited by 13
  38. 2025Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantizationcited by 13

50 most cited of 198 All 198 (JSON)