# UNIFERENCE: A Discrete Event Simulation Framework for Developing Distributed AI Models

arXiv: https://arxiv.org/abs/2603.26469 · HTML: https://mlsanity.com/paper/2603.26469/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023)
- [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024)
- [LLM in a flash: Efficient Large Language Model Inference with Limited Memory](https://mlsanity.com/paper/2312.11514/) (2023)
- [Vidur: A Large-Scale Simulation Framework For LLM Inference](https://mlsanity.com/paper/2405.05465/) (2024)
- [A Review on Edge Large Language Models: Design, Execution, and Applications](https://mlsanity.com/paper/2410.11845/) (2024)
- [Galaxy: A Resource-Efficient Collaborative Edge AI System for In-situ Transformer Inference](https://mlsanity.com/paper/2405.17245/) (2024)
- [Edge AI: A Taxonomy, Systematic Review and Future Directions](https://mlsanity.com/paper/2407.04053/) (2024)
- [Pipeline Parallelism for Inference on Heterogeneous Edge Computing](https://mlsanity.com/paper/2110.14895/) (2021)
- [DistSim: A performance model of large-scale hybrid distributed DNN training](https://mlsanity.com/paper/2306.08423/) (2023)
- [PipeFill: Using GPUs During Bubbles in Pipeline-parallel LLM Training](https://mlsanity.com/paper/2410.07192/) (2024)
- [DEFER: Distributed Edge Inference for Deep Neural Networks](https://mlsanity.com/paper/2201.06769/) (2022)
- [A Discrete-event-based Simulator for Distributed Deep Learning](https://mlsanity.com/paper/2112.00952/) (2021)
- [EdgeAISim: A Toolkit for Simulation and Modelling of AI Models in Edge Computing Environments](https://mlsanity.com/paper/2310.05605/) (2023)
- [PRISM: Distributed Inference for Foundation Models at Edge](https://mlsanity.com/paper/2507.12145/) (2025)
- [ElastiFormer: Learned Redundancy Reduction in Transformer via Self-Distillation](https://mlsanity.com/paper/2411.15281/) (2024)
- [FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inference](https://mlsanity.com/paper/2507.02620/) (2025)

## Abstract (arXiv)

Developing and evaluating distributed inference algorithms remains difficult due to the lack of standardized tools for modeling heterogeneous devices and networks. Existing studies often rely on ad-hoc testbeds or proprietary infrastructure, making results hard to reproduce and limiting exploration of hypothetical hardware or network configurations. We present UNIFERENCE, a discrete-event simulation (DES) framework designed for developing, benchmarking, and deploying distributed AI models within a unified environment. UNIFERENCE models device and network behavior through lightweight logical processes that synchronize only on communication primitives, eliminating rollbacks while preserving the causal order. It integrates seamlessly with PyTorch Distributed, enabling the same codebase to transition from simulation to real deployment. Our evaluation demonstrates that UNIFERENCE profiles runtime with up to 98.6% accuracy compared to real physical deployments across diverse backends and hardware setups. By bridging simulation and deployment, UNIFERENCE provides an accessible, reproducible platform for studying distributed inference algorithms and exploring future system designs, from high-performance clusters to edge-scale devices. The framework is open-sourced at https://github.com/Dogacel/Uniference.