mlsanityData to 14 Oct 2026

LUMOS: Democratizing SciML Workflows with L0-Regularized Learning for Unified Feature and Parameter Adaptation

arXiv 2602.22537 ↗First posted 26 Feb 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The rapid growth of scientific machine learning (SciML) has accelerated discovery across diverse domains, yet designing effective SciML models remains a challenging task. In practice, building such models often requires substantial prior knowledge and manual expertise, particularly in determining which input features to use and how large the model should be. We introduce LUMOS, an end-to-end framework based on L0-regularized learning that unifies feature selection and model pruning to democratize SciML model design. By employing semi-stochastic gating and reparameterization techniques, LUMOS dynamically selects informative features and prunes redundant parameters during training, reducing the reliance on manual tuning while maintaining predictive accuracy. We evaluate LUMOS across 13 diverse SciML workloads, including cosmology and molecular sciences, and demonstrate its effectiveness and generalizability. Experiments on 13 SciML models show that LUMOS achieves 71.45% parameter reduction and a 6.4x inference speedup on average. Furthermore, Distributed Data Parallel (DDP) training on up to eight GPUs confirms the scalability of

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 72 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

35 of 72 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  5. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  6. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  7. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  8. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  9. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
  10. 2019Similarity-Preserving Knowledge Distillationcited by 516
  11. 2023ClimaX: A foundation model for weather and climatecited by 337
  12. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 315
23 more
  1. 2022DiffDock: Diffusion Steps, Twists, and Turns for Molecular Dockingcited by 313
  2. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 301
  3. 2023DepGraph: Towards Any Structural Pruningcited by 203
  4. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 179
  5. 2018Deep learning to represent sub-grid processes in climate modelscited by 156
  6. 2022EquiBind: Geometric Deep Learning for Drug Binding Structure Predictioncited by 144
  7. 2021Group Fisher Pruning for Practical Network Compressioncited by 131
  8. 2019Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networkscited by 118
  9. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  10. 2019A Signal Propagation Perspective for Pruning Neural Networks at Initializationcited by 96
  11. 2018Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantificationcited by 81
  12. 2019Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structurecited by 76
  13. 2021Only Train Once: A One-Shot Neural Network Training And Pruning Frameworkcited by 70
  14. 2021Scientific Machine Learning Benchmarkscited by 60
  15. 2023PanGu-Sigma: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computingcited by 56
  16. 2019NAS-FCOS: Fast Neural Architecture Search for Object Detectioncited by 53
  17. 2019Learning Metrics from Teachers: Compact Networks for Image Embeddingcited by 39
  18. 2022State-specific protein-ligand complex structure prediction with a multi-scale deep generative modelcited by 35
  19. 2023Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devicescited by 17
  20. 2018CosmoFlow: Using Deep Learning to Learn the Universe at Scalecited by 14
  21. 2021Coupling streaming AI and HPC ensembles to achieve 100-1000x faster biomolecular simulationscited by 8
  22. 2024HPAC-ML: A Programming Model for Embedding ML Surrogates in Scientific Applicationscited by 1
  23. 2025k-fold Subsampling based Sequential Backward Feature Eliminationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SciML Agents: Write the Solver, Not the Solution
  2. 2025Learning to Shard: RL for Co-optimizing the Parallelism Degrees and Per-operator Sharding Dimensions in Distributed LLM Inference
  3. 2024DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
  4. 2026CoFEH: LLM-driven Feature Engineering Empowered by Collaborative Bayesian Hyperparameter Optimization
  5. 2025SciDA: Scientific Dynamic Assessor of LLMs
  6. 2026Unleashing LLMs in Bayesian Optimization: Preference-Guided Framework for Scientific Discovery
  7. 2024Natural GaLore: Accelerating GaLore for memory-efficient LLM Training and Fine-tuning
  8. 2025COSMOS: Predictable and Cost-Effective Adaptation of LLMs