mlsanityData to 7 Oct 2026

Adaptive Inference: Theoretical Limits and Unexplored Opportunities

arXiv 2402.04359 ↗First posted 6 Feb 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper introduces the first theoretical framework for quantifying the efficiency and performance gain opportunity size of adaptive inference algorithms. We provide new approximate and exact bounds for the achievable efficiency and performance gains, supported by empirical evidence demonstrating the potential for 10-100x efficiency improvements in both Computer Vision and Natural Language Processing tasks without incurring any performance penalties. Additionally, we offer insights on improving achievable efficiency gains through the optimal selection and design of adaptive inference state spaces.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 28 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

19 of 28 references.

References · 19

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  7. 2021Dynamic Neural Networks: A Surveycited by 405
  8. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 168
  9. 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 168
  10. 2019Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computingcited by 150
  11. 2020Resolution Adaptive Networks for Efficient Inferencecited by 146
  12. 2020AR-Net: Adaptive Frame Resolution for Efficient Action Recognitioncited by 88
7 more
  1. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 74
  2. 2022A Survey on Model Compression and Acceleration for Pretrained Language Modelscited by 45
  3. 2022Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Modelscited by 21
  4. 2023Adaptive Gating in Mixture-of-Experts based Language Modelscited by 16
  5. 2022AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translationcited by 9
  6. 2023Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settingscited by 5
  7. 2023Retrospective: EIE: Efficient Inference Engine on Sparse and Compressed Neural Networkcited by 2

Built on by

Not cited yet.