# Transfer Learning Beyond the Standard Model

arXiv: https://arxiv.org/abs/2510.19168 · HTML: https://mlsanity.com/paper/2510.19168/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 19 references that are in the map.
- First posted on arXiv 22 Oct 2025.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [A Survey on Negative Transfer](https://mlsanity.com/paper/2009.00909/) (2020)
- [Multilevel neural simulation-based inference](https://mlsanity.com/paper/2506.06087/) (2025)
- [Transfer learning for multifidelity simulation-based inference in cosmology](https://mlsanity.com/paper/2505.21215/) (2025)
- [Information-Ordered Bottlenecks for Adaptive Semantic Compression](https://mlsanity.com/paper/2305.11213/) (2023)
- [Simulation-Efficient Cosmological Inference with Multi-Fidelity SBI](https://mlsanity.com/paper/2507.00514/) (2025)

## Abstract (arXiv)

Machine learning enables powerful cosmological inference but typically requires many high-fidelity simulations covering many cosmological models. Transfer learning offers a way to reduce the simulation cost by reusing knowledge across models. We show that pre-training on the standard model of cosmology, LambdaCDM, and fine-tuning on various beyond-LambdaCDM scenarios -- including massive neutrinos, modified gravity, and primordial non-Gaussianities -- can enable inference with significantly fewer beyond-LambdaCDM simulations. However, we also show that negative transfer can occur when strong physical degeneracies exist between LambdaCDM and beyond-LambdaCDM parameters. We consider various transfer architectures, finding that including bottleneck structures provides the best performance. Our findings illustrate the opportunities and pitfalls of foundation-model approaches in physics: pre-training can accelerate inference, but may also hinder learning new physics.