mlsanityData to 14 Oct 2026

EPARA: Parallelizing Categorized AI Inference in Edge Clouds

arXiv 2511.00603 ↗First posted 1 Nov 2025cs.DC cs.AI cs.NIcited by 0

Not on the map. Why?

Abstract (arXiv)

With the increasing adoption of AI applications such as large language models and computer vision AI, the computational demands on AI inference systems are continuously rising, making the enhancement of task processing capacity using existing hardware a primary objective in edge clouds. We propose EPARA, an end-to-end AI parallel inference framework in edge, aimed at enhancing the edge AI serving capability. Our key idea is to categorize tasks based on their sensitivity to latency/frequency and requirement for GPU resources, thereby achieving both request-level and service-level task-resource allocation. EPARA consists of three core components: 1) a task-categorized parallelism allocator that decides the parallel mode of each task, 2) a distributed request handler that performs the calculation for the specific request, and 3) a state-aware scheduler that periodically updates service placement in edge clouds. We implement a EPARA prototype and conduct a case study on the EPARA operation for LLMs and segmentation tasks. Evaluation through testbed experiments involving edge servers, embedded devices, and microcomputers shows that EPARA achieves up to 2.1× higher goodput in production workloads compared to prior frameworks, while adapting to various edge AI inference tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 87 references that are in the map. First posted on arXiv 1 Nov 2025.

Builds on

33 of 87 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  9. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  10. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  11. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  12. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
21 more
  1. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  2. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  3. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  4. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  5. 2024YOLOv10: Real-Time End-to-End Object Detectioncited by 515
  6. 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 425
  7. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
  8. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  9. 2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 99
  10. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 75
  11. 2018Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implicationscited by 36
  12. 2022GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edgecited by 32
  13. 2023CASSINI: Network-Aware Job Scheduling in Machine Learning Clusterscited by 26
  14. 2024Galaxy: A Resource-Efficient Collaborative Edge AI System for In-situ Transformer Inferencecited by 25
  15. 2023SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentationcited by 22
  16. 2023AdaptiveNet: Post-deployment Neural Architecture Adaptation for Diverse Edge Environmentscited by 19
  17. 2025Jupiter: Fast and Resource-Efficient Collaborative Inference of Generative LLMs on Edge Devicescited by 15
  18. 2025Adaptive Semantic Token Communication for Transformer-based Edge Inferencecited by 14
  19. 2024Asteroid: Resource-Efficient Hybrid Pipeline Parallelism for Collaborative DNN Training on Heterogeneous Edge Devicescited by 13
  20. 2023SuperServe: Fine-Grained Inference Serving for Unpredictable Workloadscited by 12
  21. 2024A Survey on Integrated Sensing, Communication, and Computationcited by 12

Built on by

Not cited yet.