mlsanityData to 14 Oct 2026

Accelerated AI Inference via Dynamic Execution Methods

arXiv 2411.00853 ↗First posted 30 Oct 2024cs.LG cs.AI cs.CLcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

also in 1 more
Abstract (arXiv)

In this paper, we focus on Dynamic Execution techniques that optimize the computation flow based on input. This aims to identify simpler problems that can be solved using fewer resources, similar to human cognition. The techniques discussed include early exit from deep networks, speculative sampling for language models, and adaptive steps for diffusion models. Experimental results demonstrate that these dynamic approaches can significantly improve latency and throughput without compromising quality. When combined with model-based optimizations, such as quantization, dynamic execution provides a powerful multi-pronged strategy to optimize AI inference. Generative AI requires a large amount of compute resources. This is expected to grow, and demand for resources in data centers through to the edge is expected to continue to increase at high rates. We take advantage of existing research and provide additional innovations for some generative optimizations. In the case of LLMs, we provide more efficient sampling methods that depend on the complexity of the data. In the case of diffusion model generation, we provide a new method that also leverages the difficulty of the input prompt to predict an optimal early stopping point. Therefore, dynamic execution methods are relevant because they add another dimension of performance optimizations. Performance is critical from a competitive point of view, but increasing capacity can result in significant power savings and cost savings. We have provided several integrations of these techniques into several Intel performance libraries and Huggingface Optimum. These integrations will make them easier to use and increase the adoption of these techniques.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 20 of its 30 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

20 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Scalable Diffusion Models with Transformers19 Dec

Other references · 19

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2022Video Diffusion Modelscited by 2,305
  3. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  4. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  5. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  6. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  7. 2019Green AIcited by 627
  8. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
  9. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 370
  10. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 298
  11. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  12. 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 252
7 more
  1. 2022Diffusion Models for Video Prediction and Infillingcited by 135
  2. 2023Hierarchical Integration Diffusion Model for Realistic Image Deblurringcited by 88
  3. 2022SinFusion: Training Diffusion Models on a Single Image or Videocited by 58
  4. 2021Patterns, predictions, and actions: A story about machine learningcited by 14
  5. 2023Adaptive Deep Neural Network Inference Optimization with EENetcited by 9
  6. 2024Step Saver: Predicting Minimum Denoising Steps for Diffusion Model Image Generationcited by 2
  7. 2023Leveraging Speculative Sampling and KV-Cache Optimizations Together for Generative AI using OpenVINOcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Performance Optimization and Comparative Analysis of Generative AI Models on Advanced Accelerators
  2. 2026Evaluating Inference Compute for Generative AI: A Framework for Enterprise Workloads
  3. 2024Dynamic layer selection in decoder-only transformers
  4. 2026Towards Execution-Grounded Automated AI Research
  5. 2025Taming the Titans: A Survey of Efficient LLM Inference Serving
  6. 2024Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization
  7. 2026Experience-Driven Dynamic Exits for LLMs with Reinforcement Learning
  8. 2023Splitwise: Efficient generative LLM inference using phase splitting