mlsanityData to 14 Oct 2026

MoMa: A Modular Deep Learning Framework for Material Property Prediction

arXiv 2502.15483 ↗First posted 21 Feb 2025cs.LG cond-mat.mtrl-scicited by 0

Not on the map. Why?

Abstract (arXiv)

Deep learning methods for material property prediction have been widely explored to advance materials discovery. However, the prevailing pre-train then fine-tune paradigm often fails to address the inherent diversity and disparity of material tasks. To overcome these challenges, we introduce MoMa, a Modular framework for Materials that first trains specialized modules across a wide range of tasks and then adaptively composes synergistic modules tailored to each downstream scenario. Evaluation across 17 datasets demonstrates the superiority of MoMa, with a substantial 14% average improvement over the strongest baseline. Few-shot and continual learning experiments further highlight MoMa's potential for real-world applications. Pioneering a new paradigm of modular material learning, MoMa will be open-sourced to foster broader community collaboration.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 90 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

32 of 90 references.

References · 32

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  4. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  5. 2022Editing Models with Task Arithmeticcited by 1,120
  6. 2020Learning to Simulate Complex Physics with Graph Networkscited by 880
  7. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  8. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  9. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
  10. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  11. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
  12. 2021The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networkscited by 291
20 more
  1. 2019Label Propagation for Deep Semi-supervised Learningcited by 289
  2. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 254
  3. 2024Evolutionary Optimization of Model Merging Recipescited by 208
  4. 2024Twin-Merging: Dynamic Integration of Modular Expertise in Model Mergingcited by 116
  5. 2023Soft Merging of Experts with Adaptive Routingcited by 90
  6. 2023Modular Deep Learningcited by 83
  7. 2024Orb: A Fast, Scalable Neural Network Potentialcited by 66
  8. 2022GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasetscited by 61
  9. 2023Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivitycited by 50
  10. 2020Scalable Transfer Learning with Expert Modelscited by 49
  11. 2023From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Predictioncited by 39
  12. 2022Crystal Twins: Self-supervised Learning for Crystalline Material Property Predictioncited by 34
  13. 2024Complete and Efficient Graph Transformers for Crystal Material Property Predictioncited by 34
  14. 2022AdsorbML: A Leap in Efficiency for Adsorption Energy Calculations using Generalizable Machine Learning Potentialscited by 26
  15. 2024On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigmcited by 26
  16. 2024Configurable Foundation Models: Building LLMs from a Modular Perspectivecited by 22
  17. 2024Neuron Specialization: Leveraging intrinsic task modularity for multilingual machine translationcited by 19
  18. 2023CrysMMNet: Multimodal Representation for Crystal Property Predictioncited by 18
  19. 2024Crystalformer: Infinitely Connected Attention for Periodic Structure Encodingcited by 15
  20. 2023MixtureGrowth: Growing Neural Networks by Recombining Learned Parameterscited by 7

Built on by

Not cited yet.