# Towards Physics-Guided Foundation Models

arXiv: https://arxiv.org/abs/2502.15013 · HTML: https://mlsanity.com/paper/2502.15013/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data](https://mlsanity.com/paper/1612.08544/) (2016)
- [Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling](https://mlsanity.com/paper/1710.11431/) (2017)
- [Physics Guided RNNs for Modeling Dynamical Systems: A Case Study in Simulating Lake Temperature Profiles](https://mlsanity.com/paper/1810.13075/) (2018)
- [Geospatial foundation models for image analysis: evaluating and enhancing NASA-IBM Prithvi's domain adaptability](https://mlsanity.com/paper/2409.00489/) (2024)
- [Towards a Tighter Bound on Possible-Rendezvous Areas: Preliminary Results](https://mlsanity.com/paper/2206.12756/) (2022)
- [Spatial Computing Opportunities in Biomedical Decision Support: The Atlas-EHR Vision](https://mlsanity.com/paper/2305.09675/) (2023)
- [Eco-PiNN: A Physics-informed Neural Network for Eco-toll Estimation](https://mlsanity.com/paper/2301.05739/) (2023)
- [Towards Kriging-informed Conditional Diffusion for Regional Sea-Level Data Downscaling](https://mlsanity.com/paper/2410.15628/) (2024)
- [Reducing False Discoveries in Statistically-Significant Regional-Colocation Mining: A Summary of Results](https://mlsanity.com/paper/2407.02536/) (2024)
- [Reducing Uncertainty in Sea-level Rise Prediction: A Spatial-variability-aware Approach](https://mlsanity.com/paper/2310.15179/) (2023)

## Built on by (most cited first, 1 of 1)

- [A New Strategy for Artificial Intelligence: Training Foundation Models Directly on Human Brain Data](https://mlsanity.com/paper/2601.12053/) (2026)

## Abstract (arXiv)

Traditional foundation models are pre-trained on broad datasets to reduce the training resources (e.g., time, energy, labeled samples) needed for fine-tuning a wide range of downstream tasks. However, traditional foundation models struggle with out-of-distribution prediction and can produce outputs that are unrealistic and physically infeasible. We propose the notation of physics-guided foundation models (PGFM), that is, foundation models integrated with broad or general domain (e.g., scientific) physical knowledge applicable to a wide range of downstream tasks.