mlsanityData to 7 Oct 2026

A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platforms

arXiv 2306.15552 ↗First posted 27 Jun 2023cs.AR cs.ET cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Recent trends in deep learning (DL) have made hardware accelerators essential for various high-performance computing (HPC) applications, including image classification, computer vision, and speech recognition. This survey summarizes and classifies the most recent developments in DL accelerators, focusing on their role in meeting the performance demands of HPC applications. We explore cutting-edge approaches to DL acceleration, covering not only GPU- and TPU-based platforms but also specialized hardware such as FPGA- and ASIC-based accelerators, Neural Processing Units, open hardware RISC-V-based accelerators, and co-processors. This survey also describes accelerators leveraging emerging memory technologies and computing paradigms, including 3D-stacked Processor-In-Memory, non-volatile memories like Resistive RAM and Phase Change Memories used for in-memory computing, as well as Neuromorphic Processing Units, and Multi-Chip Module-based accelerators. Furthermore, we provide insights into emerging quantum-based accelerators and photonics. Finally, this survey categorizes the most influential architectures and technologies from recent years, offering readers a comprehensive perspective on the rapidly evolving field of deep learning acceleration.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 54 of its 226 references that are in the map. First posted on arXiv 27 Jun 2023.

Builds on

54 of 226 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  3. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  4. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  5. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  6. 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 427
  7. 2015Deep Learning with Limited Numerical Precisioncited by 368
  8. 2016Quantum Machine Learningcited by 325
  9. 2021Accelerating Sparse Deep Neural Networkscited by 213
  10. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 198
  11. 2020Binary Neural Networks: A Surveycited by 184
  12. 2016FINN: A Framework for Fast, Scalable Binarized Neural Network Inferencecited by 141
38 more
  1. 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 132
  2. 2019MLPerf Training Benchmarkcited by 114
  3. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  4. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
  5. 2020TensorFlow Quantum: A Software Framework for Quantum Machine Learningcited by 72
  6. 2019Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integrationcited by 59
  7. 2019Dissecting the Graphcore IPU Architecture via Microbenchmarkingcited by 47
  8. 2017NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Mapscited by 39
  9. 2020Accelerating Sparse DNN Models without Hardware-Support via Tile-Wise Sparsitycited by 30
  10. 2022F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantizationcited by 26
  11. 2022AI and ML Accelerator Survey and Trendscited by 23
  12. 2018UCNN: Exploiting Computational Reuse in Deep Neural Networks via Weight Repetitioncited by 19
  13. 2021Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Modecited by 16
  14. 2019MorphIC: A 65-nm 738k-Synapse/mm2 Quad-Core Binary-Weight Digital Neuromorphic Processor with Stochastic Spike-Driven Online Learningcited by 13
  15. 2023Flexagon: A Multi-Dataflow Sparse-Sparse Matrix Multiplication Accelerator for Efficient DNN Processingcited by 13
  16. 2020PERMDNN: Efficient Compressed DNN Architecture with Permuted Diagonal Matricescited by 12
  17. 2018GANAX: A Unified MIMD-SIMD Acceleration for Generative Adversarial Networkscited by 11
  18. 2020A Systematic Survey of General Sparse Matrix-Matrix Multiplicationcited by 9
  19. 2018An Energy-Efficient FPGA-based Deconvolutional Neural Networks Accelerator for Single Image Super-Resolutioncited by 8
  20. 2022RedMulE: A Compact FP16 Matrix-Multiplication Accelerator for Adaptive Deep Learning on RISC-V-Based Ultra-Low-Power SoCscited by 7
  21. 2022Kraken: A Direct Event/Frame-Based Multi-sensor Fusion SoC for Ultra-Efficient Visual Processing in Nano-UAVscited by 7
  22. 2015Origami: A 803 GOp/s/W Convolutional Network Acceleratorcited by 6
  23. 2017A Reconfigurable Streaming Deep Convolutional Neural Network Accelerator for Internet of Thingscited by 6
  24. 2020Manticore: A 4096-core RISC-V Chiplet Architecture for Ultra-efficient Floating-point Computingcited by 5
  25. 2022Neural-PIM: Efficient Processing-In-Memory with Neural Approximation of Peripheralscited by 5
  26. 2023TinyVers: A Tiny Versatile System-on-chip with State-Retentive eMRAM for ML Inference at the Extreme Edgecited by 5
  27. 2023DARKSIDE: A Heterogeneous RISC-V Compute Cluster for Extreme-Edge On-Chip DNN Inference and Trainingcited by 5
  28. 2021PIM-DRAM: Accelerating Machine Learning Workloads using Processing in Commodity DRAMcited by 4
  29. 2021PERCIVAL: Open-Source Posit RISC-V Core with Quire Capabilitycited by 4
  30. 2023RedMule: A Mixed-Precision Matrix-Matrix Operation Engine for Flexible and Energy-Efficient On-Chip Linear Algebra and TinyML Training Accelerationcited by 4
  31. 2020ESP4ML: Platform-Based Design of Systems-on-Chip for Embedded Machine Learningcited by 3
  32. 2021A Survey of Near-Data Processing Architectures for Neural Networkscited by 3
  33. 2022Dustin: A 16-Cores Parallel Ultra-Low-Power Cluster with 2b-to-32b Fully Flexible Bit-Precision and Vector Lockstep Execution Modecited by 3
  34. 2022MiniFloat-NN and ExSdotp: An ISA Extension and a Modular Open Hardware Unit for Low-Precision Training on RISC-V corescited by 3
  35. 2023SamurAI: A Versatile IoT Node With Event-Driven Wake-Up and Embedded ML Accelerationcited by 3
  36. 2019Ara: A 1 GHz+ Scalable and Energy-Efficient RISC-V Vector Processor with Multi-Precision Floating Point Support in 22 nm FD-SOIcited by 2
  37. 2022Spatz: A Compact Vector Processing Unit for High-Performance and Energy-Efficient Shared-L1 Clusterscited by 2
  38. 2022A "New Ara" for Vector Computing: An Open Source Highly Efficient RISC-V V 1.0 Vector Processor Designcited by 2

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 10; 2026: 7.

21 papers.

Sort:

By area:

  1. 2024Generative AI for the Optimization of Next-Generation Wireless Networks: Basics, State-of-the-Art, and Open Challengescited by 14
  2. 2025LILogic Net: Compact Logic Gate Networks with Learnable Connectivity for Efficient Hardware Deploymentcited by 6
  3. 2026Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Reviewcited by 4
  4. 2026Trusting What You Cannot See: Auditable Fine-Tuning and Inference for Proprietary AIcited by 3
  5. 2026Towards Cold-Start Drafting and Continual Refining: A Value-Driven Memory Approach with Application to NPU Kernel Synthesiscited by 3
  6. 2023Toward Energy-Efficient Massive MIMO: Graph Neural Network Precoding for Mitigating Non-Linear PA Distortioncited by 2
  7. 2025An Open-Source HW-SW Co-Development Framework Enabling Efficient Multi-Accelerator Systemscited by 2
  8. 2025Deep Neural Networks Inspired by Differential Equationscited by 2
  9. 2024Mathematical Algorithm Design for Deep Learning under Societal and Judicial Constraints: The Algorithmic Transparency Requirementcited by 1
  10. 2025Performance Measurements in the AI-Centric Computing Continuum Systemscited by 1
  11. 2025On the accuracy of implicit neural representations for cardiovascular anatomies and hemodynamic fieldscited by 1
  12. 2023A Survey on Design Methodologies for Accelerating Deep Learning on Heterogeneous Architecturescited by 0
9 more
  1. 2025Puzzle: Scheduling Multiple Deep Learning Models on Mobile Device with Heterogeneous Processorscited by 0
  2. 2025CURA: Size Isnt All You Need -- A Compact Universal Architecture for On-Device Intelligencecited by 0
  3. 2025Mozart: A Chiplet Ecosystem-Accelerator Codesign Framework for Composable Bespoke Application Specific Integrated Circuitscited by 0
  4. 2025ScaleDL: Towards Scalable and Efficient Runtime Prediction for Distributed Deep Learning Workloadscited by 0
  5. 2025Neural surrogates for designing gravitational wave detectorscited by 0
  6. 2026'1'-bit Count-based Sorting Unit to Reduce Link Power in DNN Acceleratorscited by 0
  7. 2026ALADIN: Accuracy-Latency-Aware Design-space Inference Analysis for Embedded AI Acceleratorscited by 0
  8. 2026AI-RAN on NPUs: Baseband Processing Without Baseband Chipscited by 0
  9. 2026From Experience to Expertise: Adoption-Aware Memory Learning for Data-Scarce NPU Kernel Synthesiscited by 0