mlsanityData to 7 Oct 2026

Successive Refinement in Large-Scale Computation: Advancing Model Inference Applications

arXiv 2402.07229 ↗First posted 11 Feb 2024cs.IT cs.AI math.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern computationally-intensive applications often operate under time constraints, necessitating acceleration methods and distribution of computational workloads across multiple entities. However, the outcome is either achieved within the desired timeline or not, and in the latter case, valuable resources are wasted. In this paper, we introduce solutions for layered-resolution computation. These solutions allow lower-resolution results to be obtained at an earlier stage than the final result. This innovation notably enhances the deadline-based systems, as if a computational job is terminated due to time constraints, an approximate version of the final result can still be generated. Moreover, in certain operational regimes, a high-resolution result might be unnecessary, because the low-resolution result may already deviate significantly from the decision threshold, for example in AI-based decision-making systems. Therefore, operators can decide whether higher resolution is needed or not based on intermediate results, enabling computations with adaptive resolution. We present our framework for two critical and computationally demanding jobs: distributed matrix multiplication (linear) and model inference in machine learning (nonlinear). Our theoretical and empirical results demonstrate that the execution delay for the first resolution is significantly shorter than that for the final resolution, while maintaining overall complexity comparable to the conventional one-shot approach. Our experiments further illustrate how the layering feature increases the likelihood of meeting deadlines and enables adaptability and transparency in massive, large-scale computations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 83 references that are in the map. First posted on arXiv 11 Feb 2024.

Builds on

43 of 83 references.

References · 43

  1. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  2. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  3. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  4. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  5. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  6. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  7. 2015Deep Learning with Limited Numerical Precisioncited by 368
  8. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  9. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  10. 2019Universally Slimmable Networks and Improved Training Techniquescited by 219
  11. 2017BlockDrop: Dynamic Inference Paths in Residual Networkscited by 185
  12. 2017Convolutional Networks with Adaptive Inference Graphscited by 147
31 more
  1. 2020Resolution Adaptive Networks for Efficient Inferencecited by 146
  2. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  3. 2020Distilling Knowledge from Graph Convolutional Networkscited by 128
  4. 2019Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inferencecited by 91
  5. 2015Speeding Up Distributed Machine Learning Using Codescited by 76
  6. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 74
  7. 2019LiteEval: A Coarse-to-Fine Framework for Resource Efficient Video Recognitioncited by 65
  8. 2021FrameExit: Conditional Early Exiting for Efficient Video Recognitioncited by 59
  9. 2020Learning When and Where to Zoom with Deep Reinforcement Learningcited by 43
  10. 2021Towards Compact CNNs via Collaborative Compressioncited by 36
  11. 2021Dynamic Network Quantization for Efficient Video Inferencecited by 36
  12. 2017Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix Multiplicationcited by 32
  13. 2020HAPI: Hardware-Aware Progressive Inferencecited by 30
  14. 2018On the Optimal Recovery Threshold of Coded Matrix Multiplicationcited by 27
  15. 2020Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Frameworkcited by 27
  16. 2021Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networkscited by 24
  17. 2023Dynamic Grained Encoder for Vision Transformerscited by 24
  18. 2017Gradient Coding from Cyclic MDS Codes and Expander Graphscited by 19
  19. 2020Fully Dynamic Inference with Deep Neural Networkscited by 19
  20. 2020Berrut Approximated Coded Computing: Straggler Resistance Beyond Polynomial Computingcited by 19
  21. 2018CodedSketch: A Coding Scheme for Distributed Computation of Approximated Matrix Multiplicationcited by 16
  22. 2019Successive Refinement of Images with Deep Joint Source-Channel Codingcited by 14
  23. 2019Not All Attention Is Needed: Gated Attention Network for Sequence Datacited by 13
  24. 2017Coded Computation over Heterogeneous Clusterscited by 12
  25. 2018A Unified Coded Deep Neural Network Training Strategy Based on Generalized PolyDot Codes for Matrix Multiplicationcited by 8
  26. 2020A Survey of Coded Distributed Computingcited by 8
  27. 2022Stream Iterative Distributed Coded Computing for Learning Applications in Heterogeneous Systemscited by 3
  28. 2017Coded Fourier Transformcited by 2
  29. 2022Distributed Computations with Layered Resolutioncited by 2
  30. 2018Near Optimal Coded Data Shuffling for Distributed Learningcited by 1
  31. 2023Randomized Polar Codes for Anytime Distributed Machine Learningcited by 1

Built on by

Not cited yet.