mlsanityData to 7 Oct 2026

Mind Mappings: Enabling Efficient Algorithm-Accelerator Mapping Space Search

arXiv 2103.01489 ↗First posted 2 Mar 2021cs.LG cs.ARcited by 24

Not on the map. Why?

Abstract (arXiv)

Modern day computing increasingly relies on specialization to satiate growing performance and efficiency requirements. A core challenge in designing such specialized hardware architectures is how to perform mapping space search, i.e., search for an optimal mapping from algorithm to hardware. Prior work shows that choosing an inefficient mapping can lead to multiplicative-factor efficiency overheads. Additionally, the search space is not only large but also non-convex and non-smooth, precluding advanced search techniques. As a result, previous works are forced to implement mapping space search using expert choices or sub-optimal search heuristics. This work proposes Mind Mappings, a novel gradient-based search method for algorithm-accelerator mapping space search. The key idea is to derive a smooth, differentiable approximation to the otherwise non-smooth, non-convex search space. With a smooth, differentiable approximation, we can leverage efficient gradient-based search algorithms to find high-quality mappings. We extensively compare Mind Mappings to black-box optimization schemes used in prior work. When tasked to find mappings for two important workloads (CNN and MTTKRP), the proposed search finds mappings that achieve an average 1.40×, 1.76×, and 1.29× (when run for a fixed number of steps) and 3.16×, 4.19×, and 2.90× (when run for a fixed amount of time) better energy-delay product (EDP) relative to Simulated Annealing, Genetic Algorithms and Reinforcement Learning, respectively. Meanwhile, Mind Mappings returns mappings with only 5.32× higher EDP than a possibly unachievable theoretical lower-bound, indicating proximity to the global optima.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 32 of its 113 references that are in the map. First posted on arXiv 2 Mar 2021.

Builds on

32 of 113 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2014Going Deeper with Convolutionscited by 4,231
  7. 2015Continuous control with deep reinforcement learningcited by 3,279
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  9. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  10. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  11. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  12. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
20 more
  1. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  2. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  3. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 291
  4. 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 132
  5. 2018An Alternative View: When Does SGD Escape Local Minima?cited by 115
  6. 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 104
  7. 2018Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractionscited by 84
  8. 2017REBAR: Low-variance, unbiased gradient estimates for discrete latent variable modelscited by 78
  9. 2015Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guaranteescited by 68
  10. 2018Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Codecited by 53
  11. 2018Ithemal: Accurate, Portable and Fast Basic Block Throughput Estimation using Deep Neural Networkscited by 43
  12. 2018SCALE-Sim: Systolic CNN Accelerator Simulatorcited by 26
  13. 2017Adversarial Variational Optimization of Non-Differentiable Simulatorscited by 22
  14. 2018UCNN: Exploiting Computational Reuse in Deep Neural Networks via Weight Repetitioncited by 19
  15. 2020DiffTune: Optimizing CPU Simulator Parameters with Learned Differentiable Surrogatescited by 14
  16. 2018GANAX: A Unified MIMD-SIMD Acceleration for Generative Adversarial Networkscited by 11
  17. 2018Morph: Flexible Acceleration for 3D CNN-based Video Understandingcited by 9
  18. 2019Reinforcement Learning and Adaptive Sampling for Optimized DNN Compilationcited by 9
  19. 2018Constructing Deep Neural Networks by Bayesian Network Structure Learningcited by 6
  20. 2020Marvel: A Data-centric Compiler for DNN Operators on Spatial Acceleratorscited by 2

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 4; 2023: 8; 2024: 4; 2025: 3.

24 papers.

Sort:

  1. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  2. 2021FLAT: An Optimized Dataflow for Mitigating Attention Bottleneckscited by 38
  3. 2021Data-Driven Offline Optimization For Architecting Hardware Acceleratorscited by 29
  4. 2021CoSA: Scheduling by Constrained Optimization for Spatial Acceleratorscited by 22
  5. 2021A Full-Stack Search Technique for Domain Optimized Deep Learning Acceleratorscited by 19
  6. 2023A Hardware Evaluation Framework for Large Language Model Inferencecited by 14
  7. 2024ConSmax: Hardware-Friendly Alternative Softmax with Learnable Parameterscited by 14
  8. 2023HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsitycited by 12
  9. 2022GRANITE: A Graph Neural Network Model for Basic Block Throughput Estimationcited by 9
  10. 2024Performance Modeling and Workload Analysis of Distributed Large Language Model Training and Inferencecited by 9
  11. 2023ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Designcited by 6
  12. 2024SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module Acceleratorscited by 6
12 more
  1. 2021Union: A Unified HW-SW Co-Design Ecosystem in MLIR for Evaluating Tensor Operations on Spatial Acceleratorscited by 5
  2. 2023Shared Memory-contention-aware Concurrent DNN Execution for Diversely Heterogeneous System-on-Chipscited by 5
  3. 2025DOSA: Differentiable Model-Based One-Loop Search for DNN Acceleratorscited by 4
  4. 2022HW-Aware Initialization of DNN Auto-Tuning to Improve Exploration Time and Robustnesscited by 3
  5. 2022DNNFuser: Generative Pre-Trained Transformer as a Generalized Mapper for Layer Fusion in DNN Acceleratorscited by 2
  6. 2022Demystifying Map Space Exploration for NPUscited by 2
  7. 2023SALSA: Simulated Annealing based Loop-Ordering Scheduler for DNN Acceleratorscited by 2
  8. 2023Target-independent XLA optimization using Reinforcement Learningcited by 1
  9. 2024Obsidian: Cooperative State-Space Exploration for Performant Inference on Secure ML Acceleratorscited by 1
  10. 2025AIRCHITECT v2: Learning the Hardware Accelerator Design Space through Unified Representationscited by 1
  11. 2023KAPLA: Pragmatic Representation and Fast Solving of Scalable NN Accelerator Dataflowcited by 0
  12. 2025Multi-Agent Reinforcement Learning for Sample-Efficient Deep Neural Network Mappingcited by 0