mlsanityData to 14 Oct 2026

Meta-Learning for Speeding Up Large Model Inference in Decentralized Environments

arXiv 2508.09194 ↗First posted 8 Aug 2025cs.LG cs.AIcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

The deployment of large-scale models, such as large language models (LLMs), incurs substantial costs due to their computational demands. To mitigate these costs and address challenges related to scalability and data security, there is a growing shift towards decentralized systems for model deployment, where choosing efficient inference acceleration schemes become crucial to manage computational resources effectively and enhance system responsiveness. In this work, we address the challenge of selecting optimal acceleration methods in decentralized systems by introducing a meta-learning-based framework. This framework automates the selection process by learning from historical performance data of various acceleration techniques across different tasks. Unlike traditional methods that rely on random selection or expert intuition, our approach systematically identifies the best acceleration strategies based on the specific characteristics of each task. We demonstrate that our meta-learning framework not only streamlines the decision-making process but also consistently outperforms conventional methods in terms of efficiency and performance. Our results highlight the potential of inference acceleration in decentralized AI systems, offering a path towards more democratic and economically feasible artificial intelligence solutions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 32 of its 52 references that are in the map. First posted on arXiv 8 Aug 2025.

Builds on

32 of 52 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  8. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  11. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  12. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
20 more
  1. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  2. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  3. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  4. 2022On Distillation of Guided Diffusion Modelscited by 725
  5. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
  6. 2022Efficiently Scaling Transformer Inferencecited by 539
  7. 2020Scaling Laws for Autoregressive Generative Modelingcited by 522
  8. 2021A White Paper on Neural Network Quantizationcited by 472
  9. 2015Deep Learning with Limited Numerical Precisioncited by 376
  10. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  11. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 341
  12. 2019A Survey on Distributed Machine Learningcited by 188
  13. 2018Learning to Optimize Tensor Programscited by 140
  14. 2023Distributed Inference and Fine-tuning of Large Language Models Over The Internetcited by 42
  15. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 28
  16. 2025Will LLMs Scaling Hit the Wall? Breaking Barriers via Distributed Resources on Massive Edge Devicescited by 9
  17. 2025Beyond A Single AI Cluster: A Survey of Decentralized LLM Trainingcited by 4
  18. 2024Model Agnostic Hybrid Sharding For Heterogeneous Distributed Inferencecited by 2
  19. 2025Adaptive Orchestration for Large-Scale Inference on Heterogeneous Accelerator Systems Balancing Cost, Performance, and Resiliencecited by 2
  20. 2025Load Balancing with Network Latencies via Distributed Gradient Descentcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Meta-Learning for Speeding Up Large Model Inference in Decentralized Environments
  2. 2026AdaptEvolve: Improving Efficiency of Evolutionary AI Agents through Adaptive Model Selection
  3. 2025An Explorative Study on Distributed Computing Techniques in Training and Inference of Large Language Models ↗
  4. 2025Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models
  5. 2024Inference Acceleration for Large Language Models on CPUs
  6. 2025Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learning
  7. 2025AdaptFlow: Adaptive Workflow Optimization via Meta-Learning
  8. 2025MetaScale: Test-Time Scaling with Evolving Meta-Thoughts