{"total":125,"rows":[{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073,"area":"moe"},{"id":"2606.19348","title":"DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence","created":"2026-04-26","cited_by_count":1040,"area":"moe"},{"id":"2403.19887","title":"Jamba: A Hybrid Transformer-Mamba Language Model","created":"2024-03-28","cited_by_count":401,"area":"moe"},{"id":"2202.08906","title":"ST-MoE: Designing Stable and Transferable Sparse Expert Models","created":"2022-02-17","cited_by_count":399,"area":"moe"},{"id":"2409.02060","title":"OLMoE: Open Mixture-of-Experts Language Models","created":"2024-09-03","cited_by_count":301,"area":"moe"},{"id":"2409.16040","title":"Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts","created":"2024-09-24","cited_by_count":287,"area":"moe"},{"id":"2505.06708","title":"Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free","created":"2025-05-10","cited_by_count":283,"area":"moe"},{"id":"2407.06204","title":"A Survey on Mixture of Experts in Large Language Models","created":"2024-06-26","cited_by_count":237,"area":"moe"},{"id":"2402.01739","title":"OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models","created":"2024-01-29","cited_by_count":187,"area":"moe"},{"id":"2406.16554","title":"LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training","created":"2024-06-24","cited_by_count":123,"area":"moe"},{"id":"2411.02265","title":"Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent","created":"2024-11-04","cited_by_count":90,"area":"moe"},{"id":"2403.07652","title":"Harder Tasks Need More Experts: Dynamic Routing in MoE Models","created":"2024-03-12","cited_by_count":72,"area":"moe"},{"id":"2407.04153","title":"Mixture of A Million Experts","created":"2024-07-04","cited_by_count":72,"area":"moe"},{"id":"2410.12013","title":"MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router","created":"2024-10-15","cited_by_count":62,"area":"moe"},{"id":"2406.06563","title":"Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models","created":"2024-06-03","cited_by_count":59,"area":"moe"},{"id":"2412.14711","title":"ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing","created":"2024-12-19","cited_by_count":54,"area":"moe"},{"id":"2407.00945","title":"Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs","created":"2024-07-01","cited_by_count":53,"area":"moe"},{"id":"2410.07524","title":"Upcycling Large Language Models into Mixture of Experts","created":"2024-10-10","cited_by_count":49,"area":"moe"},{"id":"2510.13999","title":"REAP the Experts: Why Pruning Prevails for One-Shot MoE compression","created":"2025-10-15","cited_by_count":44,"area":"moe"},{"id":"2406.05955","title":"Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters","created":"2024-06-10","cited_by_count":38,"area":"moe"},{"id":"2402.01771","title":"BlackMamba: Mixture of Experts for State-Space Models","created":"2024-02-01","cited_by_count":37,"area":"moe"},{"id":"2405.03133","title":"Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training","created":"2024-05-06","cited_by_count":37,"area":"moe"},{"id":"2608.30320","title":"On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability","created":"2026-08-31","cited_by_count":35,"area":"moe"},{"id":"2502.05172","title":"Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient","created":"2025-02-07","cited_by_count":31,"area":"moe"},{"id":"2505.11432","title":"MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production","created":"2025-05-16","cited_by_count":31,"area":"moe"},{"id":"2502.19261","title":"Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization","created":"2025-02-26","cited_by_count":24,"area":"moe"},{"id":"2503.07639","title":"Mixture of Experts Made Intrinsically Interpretable","created":"2025-03-05","cited_by_count":24,"area":"moe"},{"id":"2506.14038","title":"Load Balancing Mixture of Experts with Similarity Preserving Routers","created":"2025-06-16","cited_by_count":22,"area":"moe"},{"id":"2404.01365","title":"Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation","created":"2024-04-01","cited_by_count":21,"area":"moe"},{"id":"2410.19123","title":"Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design","created":"2024-10-24","cited_by_count":19,"area":"moe"},{"id":"2603.07685","title":"Scalable Training of Mixture-of-Experts Models with Megatron Core","created":"2026-03-08","cited_by_count":19,"area":"moe"},{"id":"2310.04361","title":"Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversion","created":"2023-10-06","cited_by_count":16,"area":"moe"},{"id":"2607.07675","title":"Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence","created":"2026-07-08","cited_by_count":15,"area":"moe"},{"id":"2502.03009","title":"Scaling Laws for Upcycling Mixture-of-Experts Language Models","created":"2025-02-05","cited_by_count":13,"area":"moe"},{"id":"2205.12399","title":"Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT","created":"2022-05-24","cited_by_count":12,"area":"moe"},{"id":"2602.17004","title":"Arcee Trinity Large Technical Report","created":"2026-02-19","cited_by_count":11,"area":"moe"},{"id":"2605.18106","title":"Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers","created":"2026-05-18","cited_by_count":11,"area":"moe"},{"id":"2504.12463","title":"Dense Backpropagation Improves Training for Sparse Mixture-of-Experts","created":"2025-04-16","cited_by_count":10,"area":"moe"},{"id":"2506.21103","title":"Learning to Skip the Middle Layers of Transformers","created":"2025-06-26","cited_by_count":10,"area":"moe"},{"id":"2411.08212","title":"PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model","created":"2024-11-12","cited_by_count":9,"area":"moe"},{"id":"2505.03531","title":"Faster MoE LLM Inference for Extremely Large Models","created":"2025-05-06","cited_by_count":9,"area":"moe"},{"id":"2505.05950","title":"FloE: On-the-Fly MoE Inference on Memory-constrained GPU","created":"2025-05-09","cited_by_count":9,"area":"moe"},{"id":"2508.05257","title":"MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs","created":"2025-08-07","cited_by_count":9,"area":"moe"},{"id":"2601.17111","title":"Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts","created":"2026-01-23","cited_by_count":9,"area":"moe"},{"id":"2405.15052","title":"Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM Training","created":"2024-05-23","cited_by_count":8,"area":"moe"},{"id":"2506.12119","title":"Mixture-of-Experts Can Surpass Dense LLMs Under Strictly Equal Resource","created":"2025-06-13","cited_by_count":8,"area":"moe"},{"id":"2508.02587","title":"Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules","created":"2025-08-04","cited_by_count":8,"area":"moe"},{"id":"2408.15901","title":"Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts","created":"2024-08-28","cited_by_count":7,"area":"moe"},{"id":"2506.09351","title":"DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts","created":"2025-06-11","cited_by_count":7,"area":"moe"},{"id":"2511.02237","title":"Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retraining","created":"2025-11-04","cited_by_count":7,"area":"moe"},{"id":"2602.09316","title":"Effective MoE-based LLM Compression by Exploiting Heterogeneous Inter-Group Experts Routing Frequency and Information Density","created":"2026-02-10","cited_by_count":7,"area":"moe"},{"id":"2604.02178","title":"The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level","created":"2026-04-02","cited_by_count":7,"area":"moe"},{"id":"2507.08771","title":"BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity","created":"2025-07-11","cited_by_count":6,"area":"moe"},{"id":"2410.03440","title":"Exploring the Benefit of Activation Sparsity in Pre-training","created":"2024-10-04","cited_by_count":5,"area":"moe"},{"id":"2506.05767","title":"dots.llm1 Technical Report","created":"2025-06-06","cited_by_count":5,"area":"moe"},{"id":"2508.18756","title":"UltraMemV2: Memory Networks Scaling to 120B Parameters with Superior Long-Context Learning","created":"2025-08-26","cited_by_count":5,"area":"moe"},{"id":"2602.14159","title":"Synergistic Intra- and Cross-Layer Regularization Losses for MoE Expert Specialization","created":"2026-02-15","cited_by_count":5,"area":"moe"},{"id":"2605.15403","title":"$\\phi$-Balancing for Mixture-of-Experts Training","created":"2026-05-14","cited_by_count":5,"area":"moe"},{"id":"2408.04278","title":"LaDiMo: Layer-wise Distillation Inspired MoEfier","created":"2024-08-08","cited_by_count":4,"area":"moe"},{"id":"2411.16205","title":"MH-MoE: Multi-Head Mixture-of-Experts","created":"2024-11-25","cited_by_count":4,"area":"moe"},{"id":"2506.07366","title":"MoE-GPS: Guidlines for Prediction Strategy for Dynamic Expert Duplication in MoE Load Balancing","created":"2025-06-09","cited_by_count":4,"area":"moe"},{"id":"2506.18145","title":"Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection","created":"2025-06-22","cited_by_count":4,"area":"moe"},{"id":"2507.15465","title":"Rethinking LLM Inference Bottlenecks: Insights from Latent Attention and Mixture-of-Experts","created":"2025-07-21","cited_by_count":4,"area":"moe"},{"id":"2510.08055","title":"From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill","created":"2025-10-09","cited_by_count":4,"area":"moe"},{"id":"2602.09721","title":"Revealing the Challenges of Attention-FFN Disaggregation for Modern MoE Models and Hardware Systems","created":"2026-02-10","cited_by_count":4,"area":"moe"},{"id":"2605.24956","title":"NITP: Next Implicit Token Prediction for LLM Pre-training","created":"2026-05-24","cited_by_count":4,"area":"moe"},{"id":"2607.14530","title":"xHC: Expanded Hyper-Connections","created":"2026-07-16","cited_by_count":4,"area":"moe"},{"id":"2508.12801","title":"Maximum Score Routing For Mixture-of-Experts","created":"2025-08-18","cited_by_count":3,"area":"moe"},{"id":"2509.10919","title":"Lightweight Metadata-Aware Mixture-of-Experts Masked Autoencoder for Earth Observation","created":"2025-09-13","cited_by_count":3,"area":"moe"},{"id":"2606.25971","title":"Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors","created":"2026-06-24","cited_by_count":3,"area":"moe"},{"id":"2506.02890","title":"Scaling Fine-Grained MoE Beyond 50B Parameters: Empirical Evaluation and Practical Insights","created":"2025-06-03","cited_by_count":2,"area":"moe"},{"id":"2512.16248","title":"Sigma-MoE-Tiny Technical Report","created":"2025-12-18","cited_by_count":2,"area":"moe"},{"id":"2512.21911","title":"Accelerate Speculative Decoding with Sparse Computation in Verification","created":"2025-12-26","cited_by_count":2,"area":"moe"},{"id":"2601.05296","title":"MoEBlaze: Breaking the Memory Wall for Efficient MoE Training on Modern GPUs","created":"2026-01-08","cited_by_count":2,"area":"moe"},{"id":"2603.21862","title":"Holistic Scaling Laws for Optimal Mixture-of-Experts Architecture Optimization","created":"2026-03-23","cited_by_count":2,"area":"moe"},{"id":"2604.00801","title":"Routing-Free Mixture-of-Experts","created":"2026-04-01","cited_by_count":2,"area":"moe"},{"id":"2605.29714","title":"Leveraging Routing Dynamics in Mixture-of-Experts Models for Efficient Language Adaptation","created":"2026-05-28","cited_by_count":2,"area":"moe"},{"id":"2608.08627","title":"UniMoMo: Expert Merging-Based MoE Acceleration for Large Recommendation Models","created":"2026-08-09","cited_by_count":2,"area":"moe"},{"id":"2407.17783","title":"How Lightweight Can A Vision Transformer Be","created":"2024-07-25","cited_by_count":1,"area":"moe"},{"id":"2412.11834","title":"Wonderful Matrices: Combining for a More Efficient and Effective Foundation Model Architecture","created":"2024-12-16","cited_by_count":1,"area":"moe"},{"id":"2508.18376","title":"DualSparse-MoE: Coordinating Tensor/Neuron-Level Sparsity with Expert Partition and Reconstruction","created":"2025-08-25","cited_by_count":1,"area":"moe"},{"id":"2601.14792","title":"Robustness of Mixtures of Experts to Feature Noise","created":"2026-01-21","cited_by_count":1,"area":"moe"},{"id":"2602.09001","title":"DirMoE: Dirichlet-routed Mixture of Experts","created":"2026-02-09","cited_by_count":1,"area":"moe"},{"id":"2605.06665","title":"UniPool: A Globally Shared Expert Pool for Mixture-of-Experts","created":"2026-05-07","cited_by_count":1,"area":"moe"},{"id":"2605.23893","title":"Complete-muE: Optimal Hyperparameter Transfer and Scaling for MoE Models","created":"2026-05-22","cited_by_count":1,"area":"moe"},{"id":"2605.28207","title":"Pruning and Distilling Mixture-of-Experts into Dense Language Models","created":"2026-05-27","cited_by_count":1,"area":"moe"},{"id":"2506.18349","title":"SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation","created":"2025-06-23","cited_by_count":0,"area":"moe"},{"id":"2507.03221","title":"Neural Inhibition Improves Dynamic Routing and Mixture of Experts","created":"2025-07-03","cited_by_count":0,"area":"moe"},{"id":"2509.22708","title":"GZSL-MoE: Apprentissage G{\\'e}n{\\'e}ralis{\\'e} Z{\\'e}ro-Shot bas{\\'e} sur le M{\\'e}lange d'Experts pour la Segmentation S{\\'e}mantique de Nuages de Points 3DAppliqu{\\'e} {\\`a} un Jeu de Donn{\\'e}es d'Environnement de Collaboration Humain-Robot","created":"2025-09-23","cited_by_count":0,"area":"moe"},{"id":"2602.05711","title":"OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale","created":"2026-02-05","cited_by_count":0,"area":"moe"},{"id":"2602.11543","title":"Pretraining A Large Language Model using Distributed GPUs: A Memory-Efficient Decentralized Paradigm","created":"2026-02-12","cited_by_count":0,"area":"moe"},{"id":"2602.15521","title":"ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns","created":"2026-02-17","cited_by_count":0,"area":"moe"},{"id":"2605.04952","title":"Adaptive Inverted-Index Routing for Granular Mixtures-of-Experts","created":"2026-05-06","cited_by_count":0,"area":"moe"},{"id":"2605.08575","title":"Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution","created":"2026-05-09","cited_by_count":0,"area":"moe"},{"id":"2605.08809","title":"SimReg: Achieving Higher Performance in the Pretraining via Embedding Similarity Regularization","created":"2026-05-09","cited_by_count":0,"area":"moe"},{"id":"2605.09403","title":"Sparsity Moves Computation: How FFN Architecture Reshapes Attention in Small Transformers","created":"2026-05-10","cited_by_count":0,"area":"moe"},{"id":"2605.10933","title":"DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices","created":"2026-05-11","cited_by_count":0,"area":"moe"},{"id":"2605.31268","title":"Mellum2 Technical Report","created":"2026-05-29","cited_by_count":0,"area":"moe"},{"id":"2606.00761","title":"Confidence-Adaptive SwiGLU for Mixture-of-Experts","created":"2026-05-30","cited_by_count":0,"area":"moe"},{"id":"2606.07414","title":"Sparsely gated tiny linear experts","created":"2026-06-05","cited_by_count":0,"area":"moe"}],"next_cursor":"100"}