{"total":109,"rows":[{"id":"2403.02310","title":"Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve","created":"2024-03-04","cited_by_count":526,"area":"inference"},{"id":"2308.16369","title":"SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills","created":"2023-08-31","cited_by_count":193,"area":"inference"},{"id":"2403.19708","title":"Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention","created":"2024-03-23","cited_by_count":163,"area":"inference"},{"id":"2308.12066","title":"Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference","created":"2023-08-23","cited_by_count":104,"area":"moe"},{"id":"2312.15234","title":"Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems","created":"2023-12-23","cited_by_count":102,"area":"inference"},{"id":"2311.12351","title":"Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey","created":"2023-11-21","cited_by_count":72,"area":null},{"id":"2507.19427","title":"Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding","created":"2025-07-25","cited_by_count":55,"area":"moe"},{"id":"2412.14219","title":"A Survey on Inference Optimization Techniques for Mixture of Experts Models","created":"2024-12-18","cited_by_count":47,"area":"moe"},{"id":"2407.12391","title":"LLM Inference Serving: Survey of Recent Advances and Opportunities","created":"2024-07-17","cited_by_count":46,"area":null},{"id":"2504.02263","title":"MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism","created":"2025-04-03","cited_by_count":32,"area":"moe"},{"id":"2408.05499","title":"LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale","created":"2024-08-10","cited_by_count":31,"area":"inference"},{"id":"2505.11432","title":"MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production","created":"2025-05-16","cited_by_count":31,"area":"moe"},{"id":"2401.08383","title":"Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference","created":"2024-01-16","cited_by_count":28,"area":"moe"},{"id":"2502.06643","title":"MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing","created":"2025-02-10","cited_by_count":26,"area":"moe"},{"id":"2502.05370","title":"Taming Latency-Memory Trade-Off in MoE-Based LLM Serving via Fine-Grained Expert Offloading","created":"2025-02-07","cited_by_count":24,"area":"moe"},{"id":"2404.19429","title":"Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping","created":"2024-04-30","cited_by_count":22,"area":"moe"},{"id":"2502.06888","title":"Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline","created":"2025-02-09","cited_by_count":21,"area":"moe"},{"id":"2410.17954","title":"ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling","created":"2024-10-23","cited_by_count":19,"area":"moe"},{"id":"2312.10868","title":"From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape","created":"2023-12-18","cited_by_count":18,"area":"moe"},{"id":"2501.10714","title":"FSMoE: A Flexible and Scalable Training System for Sparse Mixture-of-Experts Models","created":"2025-01-18","cited_by_count":17,"area":"moe"},{"id":"2504.19720","title":"Taming the Titans: A Survey of Efficient LLM Inference Serving","created":"2025-04-28","cited_by_count":16,"area":"inference"},{"id":"2503.18599","title":"Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization","created":"2025-03-24","cited_by_count":14,"area":"inference"},{"id":"2403.12900","title":"Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inference","created":"2024-03-19","cited_by_count":13,"area":null},{"id":"2502.12224","title":"Fate: Fast Edge Inference of Mixture-of-Experts Models via Cross-Layer Gate","created":"2025-02-17","cited_by_count":13,"area":"moe"},{"id":"2509.25041","title":"GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference","created":"2025-09-29","cited_by_count":13,"area":"moe"},{"id":"2401.02643","title":"Training and Serving System of Foundation Models: A Comprehensive Survey","created":"2024-01-05","cited_by_count":11,"area":null},{"id":"2503.06823","title":"eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference","created":"2025-03-10","cited_by_count":11,"area":"moe"},{"id":"2505.11415","title":"MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems","created":"2025-05-16","cited_by_count":11,"area":"moe"},{"id":"2510.02758","title":"TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling","created":"2025-10-03","cited_by_count":10,"area":"inference"},{"id":"2510.05245","title":"Stratum: System-Hardware Co-Design with Tiered Monolithic 3D-Stackable DRAM for Efficient MoE Serving","created":"2025-10-06","cited_by_count":10,"area":"moe"},{"id":"2503.04398","title":"Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling","created":"2025-03-06","cited_by_count":9,"area":"moe"},{"id":"2505.16056","title":"Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models","created":"2025-05-21","cited_by_count":8,"area":"moe"},{"id":"2509.23638","title":"LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers","created":"2025-09-28","cited_by_count":8,"area":"moe"},{"id":"2512.13525","title":"Janus: Disaggregating Attention and Experts for Scalable MoE Inference","created":"2025-12-15","cited_by_count":8,"area":"moe"},{"id":"2603.04971","title":"Mixture of Universal Experts: Scaling Virtual Width via Depth-Width Transformation","created":"2026-03-05","cited_by_count":8,"area":"moe"},{"id":"2407.00599","title":"Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules","created":"2024-06-30","cited_by_count":7,"area":"moe"},{"id":"2410.17043","title":"Optimizing Mixture-of-Experts Inference Time Combining Model Deployment and Communication Scheduling","created":"2024-10-22","cited_by_count":7,"area":"moe"},{"id":"2502.03885","title":"InfiniteHBD: Building Datacenter-Scale High-Bandwidth Domain for LLM with Optical Circuit Switching Transceivers","created":"2025-02-06","cited_by_count":7,"area":null},{"id":"2503.00392","title":"Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving","created":"2025-03-01","cited_by_count":7,"area":"inference"},{"id":"2504.03871","title":"HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs","created":"2025-04-04","cited_by_count":7,"area":"moe"},{"id":"2504.09307","title":"Lumos: Efficient Performance Modeling and Estimation for Large-scale LLM Training","created":"2025-04-12","cited_by_count":7,"area":null},{"id":"2506.12417","title":"HarMoEny: Efficient Multi-GPU Inference of MoE Models","created":"2025-06-14","cited_by_count":7,"area":"moe"},{"id":"2508.12851","title":"Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement","created":"2025-08-18","cited_by_count":7,"area":"moe"},{"id":"2503.09716","title":"MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching","created":"2025-03-12","cited_by_count":6,"area":"moe"},{"id":"2509.24626","title":"SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving","created":"2025-09-29","cited_by_count":6,"area":"inference"},{"id":"2510.05497","title":"Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference","created":"2025-10-07","cited_by_count":6,"area":"moe"},{"id":"2602.11686","title":"LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training","created":"2026-02-12","cited_by_count":6,"area":"moe"},{"id":"2504.19925","title":"SYMI: Efficient Mixture-of-Experts Training via Model and Optimizer State Decoupling","created":"2025-04-28","cited_by_count":5,"area":"moe"},{"id":"2508.17219","title":"TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving","created":"2025-08-24","cited_by_count":5,"area":"inference"},{"id":"2509.07379","title":"DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance","created":"2025-09-09","cited_by_count":5,"area":"moe"},{"id":"2512.09277","title":"Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens","created":"2025-12-10","cited_by_count":5,"area":"moe"},{"id":"2602.18755","title":"DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS","created":"2026-02-21","cited_by_count":5,"area":"inference"},{"id":"2603.09983","title":"MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios","created":"2026-02-12","cited_by_count":5,"area":"moe"},{"id":"2505.13345","title":"Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference","created":"2025-05-19","cited_by_count":4,"area":"moe"},{"id":"2507.17133","title":"BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs","created":"2025-07-23","cited_by_count":4,"area":"moe"},{"id":"2508.09591","title":"HierMoE: Accelerating MoE Training with Hierarchical Token Deduplication and Expert Swap","created":"2025-08-13","cited_by_count":4,"area":"moe"},{"id":"2511.10676","title":"Pre-Attention Expert Prediction and Prefetching for Mixture-of-Experts Large Language Models","created":"2025-11-10","cited_by_count":4,"area":"moe"},{"id":"2605.21312","title":"Frontier: Towards Comprehensive and Accurate LLM Inference Simulation","created":"2026-05-20","cited_by_count":4,"area":"inference"},{"id":"2403.07585","title":"Communication Optimization for Distributed Training: Architecture, Advances, and Opportunities","created":"2024-03-12","cited_by_count":3,"area":null},{"id":"2411.14458","title":"Improving training time and GPU utilization in geo-distributed language model training","created":"2024-11-16","cited_by_count":3,"area":null},{"id":"2501.05313","title":"Optimizing Distributed Deployment of Mixture-of-Experts Model Inference in Serverless Computing","created":"2025-01-09","cited_by_count":3,"area":"moe"},{"id":"2504.20854","title":"Towards Easy and Realistic Network Infrastructure Testing for Large-scale Machine Learning","created":"2025-04-29","cited_by_count":3,"area":null},{"id":"2505.08944","title":"Toward Cost-Efficient Serving of Mixture-of-Experts with Asynchrony","created":"2025-05-13","cited_by_count":3,"area":"moe"},{"id":"2506.15006","title":"Scaling Intelligence: Designing Data Centers for Next-Gen Language Models","created":"2025-06-17","cited_by_count":3,"area":null},{"id":"2510.14686","title":"xLLM Technical Report","created":"2025-10-16","cited_by_count":3,"area":"inference"},{"id":"2605.08639","title":"ReLibra: Routing-Replay-Guided Load Balancing for MoE Training in Reinforcement Learning","created":"2026-05-09","cited_by_count":3,"area":"moe"},{"id":"2401.16492","title":"GPU Cluster Scheduling for Network-Sensitive Deep Learning","created":"2024-01-29","cited_by_count":2,"area":null},{"id":"2503.10725","title":"Samoyeds: Accelerating MoE Models with Structured Sparsity Leveraging Sparse Tensor Cores","created":"2025-03-13","cited_by_count":2,"area":"moe"},{"id":"2508.03148","title":"Frontier: Simulating the Next Generation of LLM Inference Systems","created":"2025-08-05","cited_by_count":2,"area":"moe"},{"id":"2510.00207","title":"FlowMoE: A Scalable Pipeline Scheduling Framework for Distributed Mixture-of-Experts Training","created":"2025-09-30","cited_by_count":2,"area":"moe"},{"id":"2510.19470","title":"HybridEP: Scaling Expert Parallelism to Cross-Datacenter Scenario via Hybrid Expert/Data Transmission","created":"2025-10-22","cited_by_count":2,"area":"moe"},{"id":"2605.02960","title":"MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Serving","created":"2026-05-03","cited_by_count":2,"area":"moe"},{"id":"2411.01288","title":"Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts","created":"2024-11-02","cited_by_count":1,"area":"moe"},{"id":"2411.16786","title":"Staleness-Centric Optimizations for Parallel Diffusion MoE Inference","created":"2024-11-25","cited_by_count":1,"area":"moe"},{"id":"2511.02647","title":"Federated Attention: A Distributed Paradigm for Collaborative LLM Inference over Edge Networks","created":"2025-11-04","cited_by_count":1,"area":null},{"id":"2511.16947","title":"Fine-grained MoE Load Balancing with Linear Programming","created":"2025-11-21","cited_by_count":1,"area":"moe"},{"id":"2601.08800","title":"MixServe: An Automatic Distributed Serving System for MoE Models with Hybrid Parallelism Based on Fused Communication Algorithm","created":"2026-01-13","cited_by_count":1,"area":"moe"},{"id":"2601.22585","title":"HetCCL: Accelerating LLM Training with Heterogeneous GPUs","created":"2026-01-30","cited_by_count":1,"area":null},{"id":"2603.06350","title":"MoEless: Efficient MoE LLM Serving via Serverless Computing","created":"2026-03-06","cited_by_count":1,"area":"moe"},{"id":"2603.17456","title":"Stage-Aware Communication Scheduling for Disaggregated LLM Serving","created":"2026-03-18","cited_by_count":1,"area":"inference"},{"id":"2606.00735","title":"ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Serving","created":"2026-05-30","cited_by_count":1,"area":"moe"},{"id":"2606.26607","title":"Moebius: Serving Mixture-of-Expert Models with Seamless Runtime Parallelism Switch","created":"2026-06-25","cited_by_count":1,"area":"moe"},{"id":"2607.18002","title":"ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernels","created":"2026-07-20","cited_by_count":1,"area":"moe"},{"id":"2503.08467","title":"Accelerating MoE Model Inference with Expert Sharding","created":"2025-03-11","cited_by_count":0,"area":"moe"},{"id":"2512.24386","title":"RedunCut: Measurement-Driven Sampling and Accuracy Performance Modeling for Low-Cost Live Video Analytics","created":"2025-12-30","cited_by_count":0,"area":null},{"id":"2602.19938","title":"A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMs","created":"2026-02-23","cited_by_count":0,"area":"moe"},{"id":"2603.27624","title":"Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Scheduling","created":"2026-03-29","cited_by_count":0,"area":"moe"},{"id":"2605.05607","title":"Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs","created":"2026-05-07","cited_by_count":0,"area":"moe"},{"id":"2605.05888","title":"MoE-Hub: Taming Software Complexity for Seamless MoE Overlap with Hardware-Accelerated Communication on Multi-GPU Systems","created":"2026-05-07","cited_by_count":0,"area":"moe"},{"id":"2605.06206","title":"Federation of Experts: Communication Efficient Distributed Inference for Large Language Models","created":"2026-05-07","cited_by_count":0,"area":"moe"},{"id":"2605.19945","title":"GEM: GPU-Variability-Aware Expert to GPU Mapping for MoE Systems","created":"2026-05-19","cited_by_count":0,"area":"moe"},{"id":"2605.20982","title":"Diagnosing Overhead in Dispatch Operations: Cross-architecture Observatory","created":"2026-05-20","cited_by_count":0,"area":"moe"},{"id":"2606.21712","title":"BatchGen: An Architecture for Scalable and Efficient Batch Inference","created":"2026-06-19","cited_by_count":0,"area":null},{"id":"2607.05116","title":"Communication-Aware Placement and Pruning for Efficient Mixture-of-Experts Inference","created":"2026-07-06","cited_by_count":0,"area":"moe"},{"id":"2607.06202","title":"UBEP: Re-architecting Expert Parallelism Communication Library for Production Superpods","created":"2026-07-07","cited_by_count":0,"area":"moe"},{"id":"2607.16562","title":"AirMoE: Statistic-Augmented Over-the-Air MoE for Collaborative Intelligence","created":"2026-07-18","cited_by_count":0,"area":"moe"},{"id":"2607.17074","title":"ThAME: 3D Memory-Enabled Heterogeneous Accelerator for LLM Mixture of Experts","created":"2026-07-19","cited_by_count":0,"area":"moe"},{"id":"2607.23099","title":"Decoding the Skew: Distribution-Aware MoE Inference with Adaptive Kernel Dispatch","created":"2026-07-25","cited_by_count":0,"area":"moe"},{"id":"2608.07964","title":"EasyBalance: Cross-Layer Load Balancing in Distributed MoE Inference","created":"2026-08-08","cited_by_count":0,"area":"moe"},{"id":"2608.13962","title":"MoE Expert Execution in Disaggregated LLM Serving with a High-Bandwidth ReRAM Near-Memory Architecture","created":"2026-08-14","cited_by_count":0,"area":"moe"}],"next_cursor":"100"}