Programming with Neural Surrogates of Programs
Not on the map. Why?
Abstract (arXiv)
Surrogates, models that mimic the behavior of programs, form the basis of a variety of development workflows. We study three surrogate-based design patterns, evaluating each in case studies on a large-scale CPU simulator. With surrogate compilation, programmers develop a surrogate that mimics the behavior of a program to deploy to end-users in place of the original program. Surrogate compilation accelerates the CPU simulator under study by 1.6×. With surrogate adaptation, programmers develop a surrogate of a program then retrain that surrogate on a different task. Surrogate adaptation decreases the simulator's error by up to 50%. With surrogate optimization, programmers develop a surrogate of a program, optimize input parameters of the surrogate, then plug the optimized input parameters back into the original program. Surrogate optimization finds simulation parameters that decrease the simulator's error by 5% compared to the error induced by expert-set parameters. In this paper we formalize this taxonomy of surrogate-based design patterns. We further describe the programming methodology common to all three design patterns. Our work builds a foundation for the emerging class of workflows based on programming with surrogates of programs.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 105 references that are in the map. First posted on arXiv 12 Dec 2021.
Builds on
33 of 105 references.
References · 33
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2016Variational Inference: A Review for Statisticianscited by 1,466
- 2014How transferable are features in deep neural networks?cited by 1,216
- 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
- 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
- 2015Automatic differentiation in machine learning: a surveycited by 846
- 2020A Primer in BERTology: What we know about how BERT workscited by 807
21 more
- 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
- 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
- 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
- 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 404
- 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 291
- 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 218
- 2018Programmatically Interpretable Reinforcement Learningcited by 192
- 2018Verifiable Reinforcement Learning via Policy Extractioncited by 176
- 2016Full Resolution Image Compression with Recurrent Neural Networkscited by 170
- 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 104
- 2020Towards Learning Convolutions from Scratchcited by 46
- 2018Ithemal: Accurate, Portable and Fast Basic Block Throughput Estimation using Deep Neural Networkscited by 43
- 2019An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Modelscited by 39
- 2020Active learning of deep surrogates for PDEs: Application to metasurface designcited by 28
- 2018NEUZZ: Efficient Fuzzing with Neural Program Smoothingcited by 19
- 2019Applying machine learning to improve simulations of a chaotic dynamical system using empirical error correctioncited by 15
- 2021Accelerating SLIDE Deep Learning on Modern CPUs: Vectorization, Quantizations, Memory Optimizations, and Morecited by 15
- 2020DiffTune: Optimizing CPU Simulator Parameters with Learned Differentiable Surrogatescited by 14
- 2018Latency and Throughput Characterization of Convolutional Neural Networks for Mobile Computer Visioncited by 13
- 2018uops.info: Characterizing Latency, Throughput, and Port Usage of Instructions on Intel Microarchitecturescited by 5
- 2018Exploiting Errors for Efficiency: A Survey from Circuits to Algorithmscited by 2
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 1.
3 papers.