mlsanityData to 7 Oct 2026

Data-Driven Offline Optimization For Architecting Hardware Accelerators

arXiv 2110.11346 ↗First posted 20 Oct 2021cs.AR cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

Industry has gradually moved towards application-specific hardware accelerators in order to attain higher efficiency. While such a paradigm shift is already starting to show promising results, designers need to spend considerable manual effort and perform a large number of time-consuming simulations to find accelerators that can accelerate multiple target applications while obeying design constraints. Moreover, such a "simulation-driven" approach must be re-run from scratch every time the set of target applications or design constraints change. An alternative paradigm is to use a "data-driven", offline approach that utilizes logged simulation data, to architect hardware accelerators, without needing any form of simulations. Such an approach not only alleviates the need to run time-consuming simulation, but also enables data reuse and applies even when set of target applications changes. In this paper, we develop such a data-driven offline optimization method for designing hardware accelerators, dubbed PRIME, that enjoys all of these properties. Our approach learns a conservative, robust estimate of the desired cost function, utilizes infeasible points, and optimizes the design against this estimate without any additional simulator queries during optimization. PRIME architects accelerators -- tailored towards both single and multiple applications -- improving performance upon state-of-the-art simulation-driven methods by about 1.54x and 1.20x, while considerably reducing the required total simulation time by 93% and 99%, respectively. In addition, PRIME also architects effective accelerators for unseen applications in a zero-shot setting, outperforming simulation-based methods by 1.26x.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 38 of its 67 references that are in the map. First posted on arXiv 20 Oct 2021.

Builds on

38 of 67 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  9. 2019Searching for MobileNetV3cited by 1,984
  10. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  11. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  12. 2018Conditional Neural Processescited by 421
26 more
  1. 2018Neural Processescited by 307
  2. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  3. 2019Attentive Neural Processescited by 280
  4. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  5. 2019Conditioning by adaptive sampling for robust designcited by 110
  6. 2014Bayesian Optimization with Unknown Constraintscited by 109
  7. 2020A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latencycited by 99
  8. 2021Conservative Objective Models for Effective Offline Model-Based Optimizationcited by 87
  9. 2019Preventing Posterior Collapse with delta-VAEscited by 86
  10. 2019Model Inversion Networks for Model-Based Optimizationcited by 83
  11. 2020Chip Placement with Deep Reinforcement Learningcited by 83
  12. 2020A Better and Faster End-to-End Model for Streaming ASRcited by 68
  13. 2020Autofocused oracles for model-based designcited by 50
  14. 2018Automatically designing CNN architectures using genetic algorithm for image classificationcited by 48
  15. 2021An Evaluation of Edge TPU Accelerators for Convolutional Neural Networkscited by 39
  16. 2021Offline Model-Based Optimization via Normalized Maximum Likelihood Estimationcited by 38
  17. 2018Practical Design Space Explorationcited by 35
  18. 2020ConfuciuX: Autonomous Hardware Resource Assignment for DNN Accelerators using Reinforcement Learningcited by 35
  19. 2020Accelerator-aware Neural Network Design using AutoMLcited by 30
  20. 2020Population-Based Black-Box Optimization for Biological Sequence Designcited by 29
  21. 2020AutoDNNchip: An Automated DNN Chip Predictor and Builder for Both FPGAs and ASICscited by 26
  22. 2021Mind Mappings: Enabling Efficient Algorithm-Accelerator Mapping Space Searchcited by 24
  23. 2021Apollo: Transferable Architecture Explorationcited by 17
  24. 2020Learned Hardware/Software Co-Design of Neural Acceleratorscited by 9
  25. 2018AutoAccel: Automated Accelerator Generation and Optimization with Composable, Parallel and Pipeline Architecturecited by 5
  26. 2020Marvel: A Data-centric Compiler for DNN Operators on Spatial Acceleratorscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 5; 2024: 9; 2025: 7; 2026: 4.

29 papers.

Sort:

By area:

  1. 2023Learning Performance-Improving Code Editscited by 103
  2. 2024OmniPred: Language Models as Universal Regressorscited by 34
  3. 2022Unsupervised Learning for Combinatorial Optimization with Principled Objective Relaxationcited by 32
  4. 2022Data-Driven Offline Decision-Making via Invariant Representation Learningcited by 31
  5. 2023Searching for High-Value Molecules Using Reinforcement Learning and Transformerscited by 18
  6. 2025Offline Model-Based Optimization: Comprehensive Reviewcited by 14
  7. 2024Offline Model-Based Optimization by Learning to Rankcited by 13
  8. 2024Functional Graphical Models: Structure Enables Offline Data-Driven Optimizationcited by 11
  9. 2022GRANITE: A Graph Neural Network Model for Basic Block Throughput Estimationcited by 9
  10. 2024Adaptive Sampling of k-Space in Magnetic Resonance for Rapid Pathology Predictioncited by 7
  11. 2023ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Designcited by 6
  12. 2025Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddingscited by 6
17 more
  1. 2023Latent Conservative Objective Models for Data-Driven Crystal Structure Predictioncited by 4
  2. 2024Dynamic Co-Optimization Compiler: Leveraging Multi-Agent Reinforcement Learning for Enhanced DNN Accelerator Performancecited by 4
  3. 2024Integrated Hardware Architecture and Device Placement Searchcited by 4
  4. 2025DOSA: Differentiable Model-Based One-Loop Search for DNN Acceleratorscited by 4
  5. 2024Cliqueformer: Model-Based Optimization with Structured Transformerscited by 3
  6. 2026On the Learnability of Offline Model-Based Optimization: A Ranking Perspectivecited by 2
  7. 2024From Function to Distribution Modeling: A PAC-Generative Approach to Offline Optimizationcited by 1
  8. 2025Large Language Model as Meta-Surrogate for Data-Driven Many-Task Optimization: A Proof-of-Principle Studycited by 1
  9. 2025Learning Design-Score Manifold to Guide Diffusion Models for Offline Optimizationcited by 1
  10. 2022Multi-Agent Reinforcement Learning for Microprocessor Design Space Explorationcited by 0
  11. 2023DEAP: Design Space Exploration for DNN Accelerator Parallelismcited by 0
  12. 2024ATraDiff: Accelerating Online Reinforcement Learning with Imaginary Trajectoriescited by 0
  13. 2025Island-Based Evolutionary Computation with Diverse Surrogates and Adaptive Knowledge Transfer for High-Dimensional Data-Driven Optimizationcited by 0
  14. 2025Offline Multi-Task Multi-Objective Data-Driven Evolutionary Algorithm with Language Surrogate Model and Implicit Q-Learningcited by 0
  15. 2026Offline Materials Optimization with CliqueFlowmercited by 0
  16. 2026ChatNeuroSim: An LLM Agent Framework for Automated Compute-in-Memory Accelerator Deployment and Optimizationcited by 0
  17. 2026PerfReasoning: How Well Do LLMs Reason on Hardware Performance?cited by 0