mlsanityData to 7 Oct 2026

Mixture of experts architectures for machine learning interatomic potentials

arXiv 2603.07977 ↗First posted 9 Mar 2026physics.chem-ph cs.LG physics.comp-phcited by 1

On the map: Mixture-of-experts → MoE theory and gating functions

Abstract (arXiv)

Machine Learning Interatomic Potentials (MLIPs) enable accurate large-scale atomistic simulations, yet improving their expressive capacity efficiently remains challenging. Here we systematically investigate Mixture-of-Experts (MoE) and Mixture-of-Linear-Experts (MoLE) architectures within the DPA3 framework for MLIPs and analyze the effects of routing strategies and expert designs. We show that sparse activation combined with shared experts yields substantial performance gains, and that nonlinear MoE formulations outperform MoLE when shared experts are present, underscoring the importance of nonlinear expert specialization. Furthermore, element-wise routing consistently surpasses configuration-level routing, while global MoE routing often leads to numerical instability. The resulting element-wise MoE model consistently outperforms all DPA3-based baselines across the OMol25, OMat24, and OC20M benchmarks. Analysis of routing patterns reveals chemically interpretable expert specialization aligned with periodic-table trends, indicating that the model effectively captures element-specific chemical characteristics for precise interatomic modeling.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts. Builds on 23 of its 58 references that are in the map. First posted on arXiv 9 Mar 2026.

Builds on

23 of 58 references.

References · 23

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  7. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  8. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  9. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  10. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
  11. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  12. 2020Directional Message Passing for Molecular Graphscited by 556
11 more
  1. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 391
  2. 2020The Open Catalyst 2020 (OC20) Dataset and Community Challengescited by 246
  3. 2024A Survey on Mixture of Experts in Large Language Modelscited by 233
  4. 2017Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanicscited by 213
  5. 2020Machine Learning Force Fieldscited by 204
  6. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
  7. 2025UMA: A Family of Universal Models for Atomscited by 99
  8. 2025Learning Smooth and Expressive Interatomic Potentials for Physical Property Predictioncited by 79
  9. 2024Open Materials 2024 (OMat24) Inorganic Materials Dataset and Modelscited by 50
  10. 2023From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Predictioncited by 39
  11. 2022DPA-1: Pretraining of Attention-based Deep Potential Model for Molecular Simulationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Mixture of Experts Framework in Machine Learning Interatomic Potentials for Atomistic Simulationscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Breaking the Training Barrier of Billion-Parameter Universal Machine Learning Interatomic Potentials
  2. 2026MLIPilot: LLM-Driven Auto-Research for Machine-Learned Interatomic Potentials
  3. 2025Automated Machine Learning Pipeline: Large Language Models-Assisted Automated Dataset Generation for Training Machine-Learned Interatomic Potentials
  4. 2026Lang2MLIP: End-to-End Language-to-Machine Learning Interatomic Potential Development with Autonomous Agentic Workflows
  5. 2026El Agente Potente: High-Throughput Agentic Atomistic Simulations
  6. 2026Physics-Aligned Electronic Ground-State Learning Improves Generalization
  7. 2026Reasoning-Driven Design of Single Atom Catalysts via a Multi-Agent Large Language Model Framework
  8. 2025ChemGraph: An Agentic Framework for Computational Chemistry Workflows