# Physics-Informed Computer Vision: A Review and Perspectives

arXiv: https://arxiv.org/abs/2305.18035 · HTML: https://mlsanity.com/paper/2305.18035/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 138 of its 287 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 30 of 138)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [CLEVRER: CoLlision Events for Video REpresentation and Reasoning](https://mlsanity.com/paper/1910.01442/) (2019)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)

## Built on by (most cited first, 28 of 28)

- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [Generative Physical AI in Vision: A Survey](https://mlsanity.com/paper/2501.10928/) (2025)
- [Improving Physics-Augmented Continuum Neural Radiance Field-Based Geometry-Agnostic System Identification with Lagrangian Particle Optimization](https://mlsanity.com/paper/2406.04155/) (2024)
- [Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGC](https://mlsanity.com/paper/2502.07007/) (2025)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities](https://mlsanity.com/paper/2501.06572/) (2025)
- [Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey](https://mlsanity.com/paper/2405.01725/) (2024)
- [FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance](https://mlsanity.com/paper/2505.13437/) (2025)
- [Seeing the Wind from a Falling Leaf](https://mlsanity.com/paper/2512.00762/) (2025)
- [PINNs for Medical Image Analysis: A Survey](https://mlsanity.com/paper/2408.01026/) (2024)
- [Adapting Physics-Informed Neural Networks to Improve ODE Optimization in Mosquito Population Dynamics](https://mlsanity.com/paper/2406.05108/) (2024)
- [LLM-to-Phy3D: Physically Conform Online 3D Object Generation with LLMs](https://mlsanity.com/paper/2506.11148/) (2025)
- [Learned Bayesian Cram'er-Rao Bound for Unknown Measurement Models Using Score Neural Networks](https://mlsanity.com/paper/2502.00724/) (2025)
- [Physics-informed Partitioned Coupled Neural Operator for Complex Networks](https://mlsanity.com/paper/2410.21025/) (2024)
- [UrbanGraph: Physics-Informed Spatio-Temporal Dynamic Heterogeneous Graphs for Urban Microclimate Prediction](https://mlsanity.com/paper/2510.00457/) (2025)
- [Physical Rule-Guided Convolutional Neural Network](https://mlsanity.com/paper/2409.02081/) (2024)
- [Machine learning approaches for automatic defect detection in photovoltaic systems](https://mlsanity.com/paper/2409.16069/) (2024)
- [Survey of Deep Learning and Physics-Based Approaches in Computational Wave Imaging](https://mlsanity.com/paper/2410.08329/) (2024)
- [Optical aberrations in autonomous driving: Physics-informed parameterized temperature scaling for neural network uncertainty calibration](https://mlsanity.com/paper/2412.13695/) (2024)
- [Training-free Quantum-Inspired Image Edge Extraction Method](https://mlsanity.com/paper/2501.18929/) (2025)
- [Exploration of Hepatitis B Virus Infection Dynamics through Physics-Informed Deep Learning Approach](https://mlsanity.com/paper/2503.10708/) (2025)
- [Physical foundations for trustworthy medical imaging: a review for artificial intelligence researchers](https://mlsanity.com/paper/2505.02843/) (2025)
- [Physics-Informed Operator Learning for Hemodynamic Modeling](https://mlsanity.com/paper/2509.17293/) (2025)
- [Physics-Informed Neural Network Frameworks for the Analysis of Engineering and Biological Dynamical Systems Governed by Ordinary Differential Equations](https://mlsanity.com/paper/2511.00043/) (2025)
- [MANTA: Physics-Informed Generalized Underwater Object Tracking](https://mlsanity.com/paper/2511.23405/) (2025)
- [A Learning-based Framework for Spatial Impulse Response Compensation in 3D Photoacoustic Computed Tomography](https://mlsanity.com/paper/2601.20291/) (2026)
- [Electrostatics-Inspired Surface Reconstruction (EISR): Recovering 3D Shapes as a Superposition of Poisson's PDE Solutions](https://mlsanity.com/paper/2602.11642/) (2026)
- [Generalist Open-World Temporal Perception](https://mlsanity.com/paper/2609.06823/) (2026)
- [The limits of exactness: On the failure of automatic differentiation in physics-informed machine learning](https://mlsanity.com/paper/2609.33078/) (2026)

## Abstract (arXiv)

The incorporation of physical information in machine learning frameworks is opening and transforming many application domains. Here the learning process is augmented through the induction of fundamental knowledge and governing physical laws. In this work, we explore their utility for computer vision tasks in interpreting and understanding visual data. We present a systematic literature review of more than 250 papers on formulation and approaches to computer vision tasks guided by physical laws. We begin by decomposing the popular computer vision pipeline into a taxonomy of stages and investigate approaches to incorporate governing physical equations in each stage. Existing approaches in computer vision tasks are analyzed with regard to what governing physical processes are modeled and formulated, and how they are incorporated, i.e. modification of input data (observation bias), modification of network architectures (inductive bias), and modification of training losses (learning bias). The taxonomy offers a unified view of the application of the physics-informed capability, highlighting where physics-informed learning has been conducted and where the gaps and opportunities are. Finally, we highlight open problems and challenges to inform future research. While still in its early days, the study of physics-informed computer vision has the promise to develop better computer vision models that can improve physical plausibility, accuracy, data efficiency, and generalization in increasingly realistic applications.