{"total":220,"rows":[{"id":"2401.04088","title":"Mixtral of Experts","created":"2024-01-08","cited_by_count":1823,"area":"moe"},{"id":"2405.04434","title":"DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model","created":"2024-05-07","cited_by_count":1135,"area":"moe"},{"id":"2401.06066","title":"DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models","created":"2024-01-11","cited_by_count":867,"area":"moe"},{"id":"2201.05596","title":"DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale","created":"2022-01-14","cited_by_count":404,"area":"moe"},{"id":"2403.19887","title":"Jamba: A Hybrid Transformer-Mamba Language Model","created":"2024-03-28","cited_by_count":401,"area":"moe"},{"id":"2409.02060","title":"OLMoE: Open Mixture-of-Experts Language Models","created":"2024-09-03","cited_by_count":301,"area":"moe"},{"id":"2112.10684","title":"Efficient Large Scale Language Modeling with Mixtures of Experts","created":"2021-12-20","cited_by_count":175,"area":"moe"},{"id":"2404.15159","title":"MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts","created":"2024-04-22","cited_by_count":147,"area":"moe"},{"id":"2406.16554","title":"LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training","created":"2024-06-24","cited_by_count":123,"area":"moe"},{"id":"2411.02265","title":"Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent","created":"2024-11-04","cited_by_count":90,"area":"moe"},{"id":"2601.07372","title":"Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models","created":"2026-01-12","cited_by_count":86,"area":"moe"},{"id":"2403.07652","title":"Harder Tasks Need More Experts: Dynamic Routing in MoE Models","created":"2024-03-12","cited_by_count":72,"area":"moe"},{"id":"2407.21770","title":"MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts","created":"2024-07-31","cited_by_count":62,"area":"moe"},{"id":"2410.12013","title":"MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router","created":"2024-10-15","cited_by_count":62,"area":"moe"},{"id":"2405.16039","title":"MoEUT: Mixture-of-Experts Universal Transformers","created":"2024-05-25","cited_by_count":59,"area":"moe"},{"id":"2404.07413","title":"JetMoE: Reaching Llama2 Performance with 0.1M Dollars","created":"2024-04-11","cited_by_count":57,"area":"moe"},{"id":"2410.07348","title":"MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts","created":"2024-10-09","cited_by_count":56,"area":"moe"},{"id":"2412.14711","title":"ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing","created":"2024-12-19","cited_by_count":54,"area":"moe"},{"id":"2507.17702","title":"Towards Greater Leverage: Scaling Laws for Efficient Mixture-of-Experts Language Models","created":"2025-07-23","cited_by_count":45,"area":"moe"},{"id":"2410.11842","title":"MoH: Multi-Head Attention as Mixture-of-Head Attention","created":"2024-10-15","cited_by_count":44,"area":"moe"},{"id":"2406.05955","title":"Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters","created":"2024-06-10","cited_by_count":38,"area":"moe"},{"id":"2402.01771","title":"BlackMamba: Mixture of Experts for State-Space Models","created":"2024-02-01","cited_by_count":37,"area":"moe"},{"id":"2405.03133","title":"Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training","created":"2024-05-06","cited_by_count":37,"area":"moe"},{"id":"2502.17298","title":"Delta Decompression for MoE-based LLMs Compression","created":"2025-02-24","cited_by_count":33,"area":"moe"},{"id":"2404.05019","title":"Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts","created":"2024-04-07","cited_by_count":32,"area":"moe"},{"id":"2601.08393","title":"Controlled LLM Training on Spectral Sphere","created":"2026-01-13","cited_by_count":31,"area":"moe"},{"id":"2408.10681","title":"HMoE: Heterogeneous Mixture of Experts for Language Modeling","created":"2024-08-20","cited_by_count":29,"area":"moe"},{"id":"2411.15708","title":"LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training","created":"2024-11-24","cited_by_count":26,"area":"moe"},{"id":"2511.00279","title":"LongCat-Flash-Omni Technical Report","created":"2025-10-31","cited_by_count":26,"area":"moe"},{"id":"2509.10377","title":"Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs","created":"2025-09-12","cited_by_count":25,"area":"moe"},{"id":"2210.16433","title":"Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models","created":"2022-10-28","cited_by_count":24,"area":"moe"},{"id":"2406.02500","title":"Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques","created":"2024-06-04","cited_by_count":24,"area":"moe"},{"id":"2406.12034","title":"Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts","created":"2024-06-17","cited_by_count":23,"area":"moe"},{"id":"2502.16894","title":"Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment","created":"2025-02-24","cited_by_count":23,"area":"moe"},{"id":"2401.02731","title":"Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks","created":"2024-01-05","cited_by_count":22,"area":"moe"},{"id":"2503.05139","title":"Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs","created":"2025-03-07","cited_by_count":22,"area":"moe"},{"id":"2506.14038","title":"Load Balancing Mixture of Experts with Similarity Preserving Routers","created":"2025-06-16","cited_by_count":22,"area":"moe"},{"id":"2404.01365","title":"Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation","created":"2024-04-01","cited_by_count":21,"area":"moe"},{"id":"2408.06793","title":"Layerwise Recurrent Router for Mixture-of-Experts","created":"2024-08-13","cited_by_count":21,"area":"moe"},{"id":"2410.19034","title":"Mixture of Parrots: Experts improve memorization more than reasoning","created":"2024-10-24","cited_by_count":21,"area":"moe"},{"id":"2512.14080","title":"SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations","created":"2025-12-16","cited_by_count":21,"area":"moe"},{"id":"2504.15299","title":"D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving","created":"2025-04-17","cited_by_count":20,"area":"moe"},{"id":"2410.07739","title":"SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture","created":"2024-10-10","cited_by_count":19,"area":"moe"},{"id":"2410.18035","title":"MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning","created":"2024-10-23","cited_by_count":19,"area":"moe"},{"id":"2405.03131","title":"WDMoE: Wireless Distributed Large Language Models with Mixture of Experts","created":"2024-05-06","cited_by_count":18,"area":"moe"},{"id":"2505.21411","title":"Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity","created":"2025-05-27","cited_by_count":18,"area":"moe"},{"id":"2508.01625","title":"EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models","created":"2025-08-03","cited_by_count":18,"area":"moe"},{"id":"2609.01343","title":"SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers","created":"2026-09-01","cited_by_count":18,"area":"moe"},{"id":"2507.23279","title":"Unveiling Super Experts in Mixture-of-Experts Large Language Models","created":"2025-07-31","cited_by_count":17,"area":"moe"},{"id":"2408.08274","title":"BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts","created":"2024-08-15","cited_by_count":16,"area":"moe"},{"id":"2503.05066","title":"Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Experts","created":"2025-03-07","cited_by_count":15,"area":"moe"},{"id":"2507.07024","title":"FlexOlmo: Open Language Models for Flexible Data Use","created":"2025-07-09","cited_by_count":15,"area":"moe"},{"id":"2501.10375","title":"DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference","created":"2024-12-16","cited_by_count":14,"area":"moe"},{"id":"2505.00315","title":"Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing","created":"2025-05-01","cited_by_count":14,"area":"moe"},{"id":"2502.03009","title":"Scaling Laws for Upcycling Mixture-of-Experts Language Models","created":"2025-02-05","cited_by_count":13,"area":"moe"},{"id":"2507.00390","title":"MoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoE","created":"2025-07-01","cited_by_count":13,"area":"moe"},{"id":"2205.12701","title":"Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Experts","created":"2022-05-25","cited_by_count":12,"area":"moe"},{"id":"2408.03511","title":"MoExtend: Tuning New Experts for Modality and Task Extension","created":"2024-08-07","cited_by_count":12,"area":"moe"},{"id":"2411.06681","title":"WDMoE: Wireless Distributed Mixture of Experts for Large Language Models","created":"2024-11-11","cited_by_count":12,"area":"moe"},{"id":"2503.15798","title":"Mixture of Lookup Experts","created":"2025-03-20","cited_by_count":12,"area":"moe"},{"id":"2504.00661","title":"DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism","created":"2025-04-01","cited_by_count":11,"area":"moe"},{"id":"2508.12461","title":"Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI's Latest Open Source Models","created":"2025-08-17","cited_by_count":11,"area":"moe"},{"id":"2511.19822","title":"Mosaic Pruning: A Hierarchical Framework for Generalizable Pruning of Mixture-of-Experts Models","created":"2025-11-25","cited_by_count":11,"area":"moe"},{"id":"2601.00583","title":"HFedMoE: Resource-aware Heterogeneous Federated Learning with Mixture-of-Experts","created":"2026-01-02","cited_by_count":11,"area":"moe"},{"id":"2407.11030","title":"DLO: Dynamic Layer Operation for Efficient Vertical Scaling of LLMs","created":"2024-07-03","cited_by_count":10,"area":"moe"},{"id":"2412.00069","title":"Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning","created":"2024-11-26","cited_by_count":10,"area":"moe"},{"id":"2501.15316","title":"ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning","created":"2025-01-25","cited_by_count":10,"area":"moe"},{"id":"2506.13329","title":"EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization","created":"2025-06-16","cited_by_count":10,"area":"moe"},{"id":"2510.00844","title":"Learning Compact Representations of LLM Abilities via Item Response Theory","created":"2025-10-01","cited_by_count":10,"area":"moe"},{"id":"2510.14436","title":"MergeMoE: Efficient Compression of MoE Models via Expert Output Merging","created":"2025-10-16","cited_by_count":10,"area":"moe"},{"id":"2510.17483","title":"ReXMoE: Reusing Experts with Minimal Overhead in Mixture-of-Experts","created":"2025-10-20","cited_by_count":10,"area":"moe"},{"id":"2606.04438","title":"LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling","created":"2026-06-03","cited_by_count":10,"area":"moe"},{"id":"2407.09816","title":"MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts","created":"2024-07-13","cited_by_count":9,"area":"moe"},{"id":"2506.18945","title":"Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models","created":"2025-06-23","cited_by_count":9,"area":"moe"},{"id":"2511.04805","title":"PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference","created":"2025-11-06","cited_by_count":9,"area":"moe"},{"id":"2203.06850","title":"Efficient Language Modeling with Sparse all-MLP","created":"2022-03-14","cited_by_count":8,"area":"moe"},{"id":"2405.15052","title":"Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM Training","created":"2024-05-23","cited_by_count":8,"area":"moe"},{"id":"2407.21571","title":"PMoE: Progressive Mixture of Experts with Asymmetric Transformer for Continual Learning","created":"2024-07-31","cited_by_count":8,"area":"moe"},{"id":"2502.04416","title":"Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis","created":"2025-02-06","cited_by_count":8,"area":"moe"},{"id":"2504.02658","title":"MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensators","created":"2025-04-03","cited_by_count":8,"area":"moe"},{"id":"2506.12119","title":"Mixture-of-Experts Can Surpass Dense LLMs Under Strictly Equal Resource","created":"2025-06-13","cited_by_count":8,"area":"moe"},{"id":"2508.02587","title":"Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules","created":"2025-08-04","cited_by_count":8,"area":"moe"},{"id":"2408.15901","title":"Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts","created":"2024-08-28","cited_by_count":7,"area":"moe"},{"id":"2501.13074","title":"Autonomy-of-Experts Models","created":"2025-01-22","cited_by_count":7,"area":"moe"},{"id":"2504.07964","title":"C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization for Test-Time Expert Re-Mixing","created":"2025-04-10","cited_by_count":7,"area":"moe"},{"id":"2508.02322","title":"CAMERA: Multi-Matrix Joint Compression for MoE Models via Micro-Expert Redundancy Analysis","created":"2025-08-04","cited_by_count":7,"area":"moe"},{"id":"2510.05781","title":"Mixture of Neuron Experts","created":"2025-10-07","cited_by_count":7,"area":"moe"},{"id":"2511.07419","title":"Routing Manifold Alignment Improves Generalization of Mixture-of-Experts LLMs","created":"2025-11-10","cited_by_count":7,"area":"moe"},{"id":"2602.09316","title":"Effective MoE-based LLM Compression by Exploiting Heterogeneous Inter-Group Experts Routing Frequency and Information Density","created":"2026-02-10","cited_by_count":7,"area":"moe"},{"id":"2605.23078","title":"GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs","created":"2026-05-21","cited_by_count":7,"area":"moe"},{"id":"2606.04980","title":"AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization","created":"2026-06-03","cited_by_count":7,"area":"moe"},{"id":"2411.00918","title":"LIBMoE: A Library for comprehensive benchmarking Mixture of Experts in Large Language Models","created":"2024-11-01","cited_by_count":6,"area":"moe"},{"id":"2503.05447","title":"Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts","created":"2025-03-07","cited_by_count":6,"area":"moe"},{"id":"2504.01337","title":"Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Design","created":"2025-04-02","cited_by_count":6,"area":"moe"},{"id":"2507.08771","title":"BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity","created":"2025-07-11","cited_by_count":6,"area":"moe"},{"id":"2509.25913","title":"Understanding the Mixture-of-Experts with Nadaraya-Watson Kernel","created":"2025-09-30","cited_by_count":6,"area":"moe"},{"id":"2606.05688","title":"Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models","created":"2026-06-04","cited_by_count":6,"area":"moe"},{"id":"2310.15961","title":"Mixture of Tokens: Continuous MoE through Cross-Example Aggregation","created":"2023-10-24","cited_by_count":5,"area":"moe"},{"id":"2409.12210","title":"Mixture of Diverse Size Experts","created":"2024-09-18","cited_by_count":5,"area":"moe"},{"id":"2410.16077","title":"CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts","created":"2024-10-21","cited_by_count":5,"area":"moe"}],"next_cursor":"100"}