ACRoBat: Optimizing Auto-batching of Dynamic Deep Learning at Compile Time
Not on the map. Why?
Abstract (arXiv)
Dynamic control flow is an important technique often used to design expressive and efficient deep learning computations for applications such as text parsing, machine translation, exiting early out of deep models and so on. The control flow divergence resulting from dynamic control flow makes batching, an important optimization enabling high throughput and hardware utilization, difficult to perform manually. In this paper, we present ACRoBat, a framework that enables efficient automatic batching for dynamic deep learning computations by performing hybrid static+dynamic compiler optimizations and end-to-end tensor code generation. ACRoBat performs up to 8.5X better than DyNet, a state-of-the-art framework for automatic batching, on an Nvidia GeForce GPU.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 80 references that are in the map. First posted on arXiv 17 May 2023.
Builds on
32 of 80 references.
References · 32
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2015Fast R-CNNcited by 2,933
- 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
- 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
- 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
- 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
- 2014cuDNN: Efficient Primitives for Deep Learningcited by 245
20 more
- 2019Depth-Adaptive Transformercited by 236
- 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 201
- 2016Sequence-to-Sequence Learning as Beam-Search Optimizationcited by 127
- 2018Tree-to-tree Neural Networks for Program Translationcited by 93
- 2020MLIR: A Compiler Infrastructure for the End of Moore's Lawcited by 65
- 2019Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoderscited by 60
- 2015Transition-Based Dependency Parsing with Stack Long Short-Term Memorycited by 45
- 2022ByteTransformer: A High-Performance Transformer Boosted for Variable-Length Inputscited by 32
- 2017DyNet: The Dynamic Neural Network Toolkitcited by 21
- 2020Cortex: A Compiler for Recursive Deep Learning Modelscited by 16
- 2017Deep Learning with Dynamic Computation Graphscited by 15
- 2021LazyTensor: combining eager execution with domain-specific compilerscited by 11
- 2015DAG-Recurrent Neural Networks For Scene Labelingcited by 10
- 2017On-the-fly Operation Batching in Dynamic Computation Graphscited by 7
- 2018Shallow-Deep Networks: Understanding and Mitigating Network Overthinkingcited by 7
- 2019Automatically Batching Control-Intensive Programs for Modern Acceleratorscited by 6
- 2018JANUS: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative Programscited by 4
- 2019Auto-Vectorizing TensorFlow Graphs: Jacobians, Auto-Batching And Beyondcited by 3
- 2020E-BATCH: Energy-Efficient and High-Throughput RNN Batchingcited by 3
- 2019Just-in-Time Dynamic-Batchingcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.