YFlows: Systematic Dataflow Exploration and Code Generation for Efficient Neural Network Inference using SIMD Architectures on CPUs
Not on the map. Why?
Abstract (arXiv)
We address the challenges associated with deploying neural networks on CPUs, with a particular focus on minimizing inference time while maintaining accuracy. Our novel approach is to use the dataflow (i.e., computation order) of a neural network to explore data reuse opportunities using heuristic-guided analysis and a code generation framework, which enables exploration of various Single Instruction, Multiple Data (SIMD) implementations to achieve optimized neural network execution. Our results demonstrate that the dataflow that keeps outputs in SIMD registers while also maximizing both input and weight reuse consistently yields the best performance for a wide variety of inference workloads, achieving up to 3x speedup for 8-bit neural networks, and up to 4.8x speedup for binary neural networks, respectively, over the optimized implementations of neural networks today.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 54 of its 118 references that are in the map. First posted on arXiv 1 Oct 2023.
Builds on
54 of 118 references.
References · 54
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
- 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
- 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
- 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
- 2021Transformers in Vision: A Surveycited by 1,009
- 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
38 more
- 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
- 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 427
- 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 291
- 2017Train longer, generalize better: closing the generalization gap in large batch training of neural networkscited by 278
- 2021Post-Training Quantization for Vision Transformercited by 274
- 2022Transformers in Medical Imaging: A Surveycited by 245
- 2020Attention is All You Need in Speech Separationcited by 184
- 2021Block Pruning For Faster Transformerscited by 176
- 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
- 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
- 2020HAT: Hardware-Aware Transformers for Efficient Natural Language Processingcited by 136
- 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 132
- 2020Data Movement Is All You Need: A Case Study on Optimizing Transformerscited by 108
- 2021Transformer Embeddings of Irregularly Spaced Events and Their Participantscited by 90
- 2018Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractionscited by 84
- 2021Vision Transformer Pruningcited by 77
- 2022DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformerscited by 65
- 2014Rigid-Motion Scattering for Texture Classificationcited by 52
- 2016Towards the Limit of Network Quantizationcited by 49
- 2022DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generationcited by 46
- 2022ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Designcited by 41
- 2019NeuroVectorizer: End-to-End Vectorization with Deep Reinforcement Learningcited by 37
- 2019Fully Quantized Transformer for Machine Translationcited by 36
- 2018Interstellar: Using Halide's Scheduling Language to Analyze DNN Acceleratorscited by 33
- 2021AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Modelscited by 31
- 2018Optimizing CNN Model Inference on CPUscited by 29
- 2019daBNN: A Super Fast Inference Framework for Binary Neural Networks on ARM devicescited by 27
- 2020Ordering Chaos: Memory-Aware Scheduling of Irregularly Wired Neural Networks for Edge Devicescited by 27
- 2021UniNet: Unified Architecture Search with Convolution, Transformer, and MLPcited by 23
- 2018Efficient Sparse-Winograd Convolutional Neural Networkscited by 20
- 2020Procrustes: a Dataflow and Accelerator for Sparse Deep Neural Network Trainingcited by 18
- 2022TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentationcited by 17
- 2021Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classificationcited by 14
- 2020Extremely Low Bit Transformer Quantization for On-Device Neural Machine Translationcited by 13
- 2020Searching for Winograd-aware Quantized Networkscited by 11
- 2022SALO: An Efficient Spatial Accelerator Enabling Hybrid Sparse Attention Mechanisms for Long Sequencescited by 10
- 2022Winograd Convolution for Deep Neural Networks: Efficient Point Selectioncited by 8
- 2021Optimizing Inference Performance of Transformers on CPUscited by 7
50 most cited of 54 All 54 (JSON)
Built on by
Citing papers per year: 2023: 1.
1 paper.