mlsanityData to 14 Oct 2026

A Survey on Design Methodologies for Accelerating Deep Learning on Heterogeneous Architectures

arXiv 2311.17815 ↗First posted 29 Nov 2023cs.AR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Given their increasing size and complexity, the need for efficient execution of deep neural networks has become increasingly pressing in the design of heterogeneous High-Performance Computing (HPC) and edge platforms, leading to a wide variety of proposals for specialized deep learning architectures and hardware accelerators. The design of such architectures and accelerators requires a multidisciplinary approach combining expertise from several areas, from machine learning to computer architecture, low-level hardware design, and approximate computing. Several methodologies and tools have been proposed to improve the process of designing accelerators for deep learning, aimed at maximizing parallelism and minimizing data movement to achieve high performance and energy efficiency. This paper critically reviews influential tools and design methodologies for Deep Learning accelerators, offering a wide perspective in this rapidly evolving field. This work complements surveys on architectures and accelerators by covering hardware-software co-design, automated synthesis, domain-specific compilers, design space exploration, modeling, and simulation, providing insights into technical challenges and open research directions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 216 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

36 of 216 references.

References · 36

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  5. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  6. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  7. 2020DAPPLE: A Pipelined Data Parallel Approach for Training Large Modelscited by 101
  8. 2017Device Placement Optimization with Reinforcement Learningcited by 86
  9. 2018Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractionscited by 85
  10. 2021Machine Learning for Electronic Design Automation: A Surveycited by 72
  11. 2023A Comprehensive Survey on Applications of Transformers for Deep Learning Taskscited by 71
  12. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 65
24 more
  1. 2019Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integrationcited by 63
  2. 2019PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inferencecited by 61
  3. 2020DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUscited by 58
  4. 2019Hardware/Software Co-Exploration of Neural Architecturescited by 50
  5. 2021Torch.fx: Practical Program Capture and Transformation for Deep Learning in Pythoncited by 40
  6. 2018Interstellar: Using Halide's Scheduling Language to Analyze DNN Acceleratorscited by 34
  7. 2020Best of Both Worlds: AutoML Codesign of a CNN and its Hardware Acceleratorcited by 30
  8. 2018Relay: A New IR for Machine Learning Frameworkscited by 24
  9. 2018Exploring Hidden Dimensions in Parallelizing Convolutional Neural Networkscited by 23
  10. 2021ScaleHLS: A New Scalable High-Level Synthesis Framework on Multi-Level Intermediate Representationcited by 21
  11. 2023A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platformscited by 21
  12. 2020Compiling ONNX Neural Network Models Using MLIRcited by 18
  13. 2022Sparseloop: An Analytical Approach To Sparse Tensor Accelerator Modelingcited by 15
  14. 2022DeFiNES: Enabling Fast Exploration of the Depth-first Scheduling Space for DNN Accelerators through Analytical Modelingcited by 12
  15. 2023Toward Sustainable HPC: Carbon Footprint Estimation and Environmental Implications of HPC Systemscited by 12
  16. 2023Relax: Composable Abstractions for End-to-End Dynamic Machine Learningcited by 12
  17. 2020The Effects of Approximate Multiplication on Convolutional Neural Networkscited by 10
  18. 2023HIDA: A Hierarchical Dataflow Compiler for High-Level Synthesiscited by 9
  19. 2017The TensorFlow Partitioning and Scheduling Problem: It's the Critical Path!cited by 4
  20. 2022QAPPA: Quantization-Aware Power, Performance, and Area Modeling of DNN Acceleratorscited by 4
  21. 2022QADAM: Quantization-Aware DNN Accelerator Modeling for Pareto-Optimalitycited by 3
  22. 2020New Approximate Multiplier for Low Power Digital Signal Processingcited by 2
  23. 2020Evaluating FPGA Accelerator Performance with a Parameterized OpenCL Adaptation of the HPCChallenge Benchmark Suitecited by 1
  24. 2020Transaction-level Model Simulator for Communication-Limited Acceleratorscited by 1

Built on by

Not cited yet.