# Orthogonal projection-based regularization for efficient model augmentation

arXiv: https://arxiv.org/abs/2501.05842 · HTML: https://mlsanity.com/paper/2501.05842/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 10 Jan 2025.

## Builds on (most cited first, 17 of 17)

- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data](https://mlsanity.com/paper/1612.08544/) (2016)
- [Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling](https://mlsanity.com/paper/1710.11431/) (2017)
- [Nonlinear System Identification: A User-Oriented Roadmap](https://mlsanity.com/paper/1902.00683/) (2019)
- [Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modeling](https://mlsanity.com/paper/2109.06407/) (2021)
- [Nonlinear state-space identification using deep encoder networks](https://mlsanity.com/paper/2012.07697/) (2020)
- [On the smoothness of nonlinear system identification](https://mlsanity.com/paper/1905.00820/) (2019)
- [Deep Subspace Encoders for Nonlinear System Identification](https://mlsanity.com/paper/2210.14816/) (2022)
- [On the Initialization of Nonlinear LFR Model Identification with the Best Linear Approximation](https://mlsanity.com/paper/2004.05040/) (2020)
- [Physics-Guided Neural Networks for Feedforward Control: An Orthogonal Projection-Based Approach](https://mlsanity.com/paper/2201.03308/) (2022)
- [Physics-guided neural networks for feedforward control with input-to-state stability guarantees](https://mlsanity.com/paper/2301.08568/) (2023)
- [On feedforward control using physics-guided neural networks: Training cost regularization and optimized initialization](https://mlsanity.com/paper/2201.12088/) (2022)
- [Multi-Objective Physics-Guided Recurrent Neural Networks for Identifying Non-Autonomous Dynamical Systems](https://mlsanity.com/paper/2204.12972/) (2022)
- [Learning-based model augmentation with LFRs](https://mlsanity.com/paper/2404.01901/) (2024)
- [Learning for Precision Motion of an Interventional X-ray System: Add-on Physics-Guided Neural Network Feedforward Control](https://mlsanity.com/paper/2303.07994/) (2023)
- [Initialization Approach for Nonlinear State-Space Identification via the Subspace Encoder Approach](https://mlsanity.com/paper/2304.02119/) (2023)

## Built on by (most cited first, 1 of 1)

- [OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems](https://mlsanity.com/paper/2606.19145/) (2026)

## Abstract (arXiv)

Deep-learning-based nonlinear system identification has shown the ability to produce reliable and highly accurate models in practice. However, these black-box models lack physical interpretability, and a considerable part of the learning effort is often spent on capturing already expected/known behavior of the system, that can be accurately described by first-principles laws of physics. A potential solution is to directly integrate such prior physical knowledge into the model structure, combining the strengths of physics-based modeling and deep-learning-based identification. The most common approach is to use an additive model augmentation structure, where the physics-based and the machine-learning (ML) components are connected in parallel, i.e., additively. However, such models are overparametrized, training them is challenging, potentially causing the physics-based part to lose interpretability. To overcome this challenge, this paper proposes an orthogonal projection-based regularization technique to enhance parameter learning and even model accuracy in learning-based augmentation of nonlinear baseline models.