mlsanityData to 7 Oct 2026

UNIFERENCE: A Discrete Event Simulation Framework for Developing Distributed AI Models

arXiv 2603.26469 ↗First posted 27 Mar 2026cs.DC cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Developing and evaluating distributed inference algorithms remains difficult due to the lack of standardized tools for modeling heterogeneous devices and networks. Existing studies often rely on ad-hoc testbeds or proprietary infrastructure, making results hard to reproduce and limiting exploration of hypothetical hardware or network configurations. We present UNIFERENCE, a discrete-event simulation (DES) framework designed for developing, benchmarking, and deploying distributed AI models within a unified environment. UNIFERENCE models device and network behavior through lightweight logical processes that synchronize only on communication primitives, eliminating rollbacks while preserving the causal order. It integrates seamlessly with PyTorch Distributed, enabling the same codebase to transition from simulation to real deployment. Our evaluation demonstrates that UNIFERENCE profiles runtime with up to 98.6% accuracy compared to real physical deployments across diverse backends and hardware setups. By bridging simulation and deployment, UNIFERENCE provides an accessible, reproducible platform for studying distributed inference algorithms and exploring future system designs, from high-performance clusters to edge-scale devices. The framework is open-sourced at https://github.com/Dogacel/Uniference.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 27 Mar 2026.

Builds on

18 of 27 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  4. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  5. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 170
  6. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 115
  7. 2024A Review on Edge Large Language Models: Design, Execution, and Applicationscited by 98
  8. 2024Galaxy: A Resource-Efficient Collaborative Edge AI System for In-situ Transformer Inferencecited by 25
  9. 2024Edge AI: A Taxonomy, Systematic Review and Future Directionscited by 21
  10. 2021Pipeline Parallelism for Inference on Heterogeneous Edge Computingcited by 13
  11. 2023DistSim: A performance model of large-scale hybrid distributed DNN trainingcited by 7
  12. 2024PipeFill: Using GPUs During Bubbles in Pipeline-parallel LLM Trainingcited by 5
6 more
  1. 2022DEFER: Distributed Edge Inference for Deep Neural Networkscited by 3
  2. 2021A Discrete-event-based Simulator for Distributed Deep Learningcited by 2
  3. 2023EdgeAISim: A Toolkit for Simulation and Modelling of AI Models in Edge Computing Environmentscited by 2
  4. 2025PRISM: Distributed Inference for Foundation Models at Edgecited by 2
  5. 2024ElastiFormer: Learned Redundancy Reduction in Transformer via Self-Distillationcited by 1
  6. 2025FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inferencecited by 1

Built on by

Not cited yet.