mlsanityData to 7 Oct 2026

Debunking the CUDA Myth Towards GPU-based AI Systems

arXiv 2501.00210 ↗First posted 31 Dec 2024cs.DC cs.AI cs.ARcited by 2

Not on the map. Why?

Abstract (arXiv)

This paper presents a comprehensive evaluation of Intel Gaudi NPUs as an alternative to NVIDIA GPUs, which is currently the de facto standard in AI system design. First, we create a suite of microbenchmarks to compare Intel Gaudi-2 with NVIDIA A100, showing that Gaudi-2 achieves competitive performance not only in primitive AI compute, memory, and communication operations but also in executing several important AI workloads end-to-end. We then assess Gaudi NPU's programmability by discussing several software-level optimization strategies to employ for implementing critical FBGEMM operators and vLLM, evaluating their efficiency against GPU-optimized counterparts. Results indicate that Gaudi-2 achieves energy efficiency comparable to A100, though there are notable areas for improvement in terms of software maturity. Overall, we conclude that, with effective integration into high-level AI frameworks, Gaudi NPUs could challenge NVIDIA GPU's dominance in the AI server market, though further improvements are necessary to fully compete with NVIDIA's robust software ecosystem.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 21 of its 59 references that are in the map. First posted on arXiv 31 Dec 2024.

Builds on

21 of 59 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  9. 2018Self-Attentive Sequential Recommendationcited by 1,169
  10. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  11. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  12. 2020DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systemscited by 326
9 more
  1. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 198
  2. 2018NVIDIA Tensor Core Programmability, Performance & Precisioncited by 77
  3. 2019RecNMP: Accelerating Personalized Recommendation with Near-Memory Processingcited by 49
  4. 2019TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learningcited by 42
  5. 2018Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarkingcited by 35
  6. 2019Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUscited by 10
  7. 2022DiVa: An Accelerator for Differentially Private Machine Learningcited by 3
  8. 2023Benchmarking and In-depth Performance Study of Large Language Models on Habana Gaudi Processorscited by 2
  9. 2024Control Flow Management in Modern GPUscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025An Inquiry into Datacenter TCO for LLM Inference with FP8cited by 4
  2. 2025BLaST: High Performance Inference and Pretraining using BLock Sparse Transformerscited by 2