# Physical Data Embedding for Memory Efficient AI

arXiv: https://arxiv.org/abs/2407.14504 · HTML: https://mlsanity.com/paper/2407.14504/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning](https://mlsanity.com/paper/2104.07857/) (2021)
- [Sustainable AI: Environmental Implications, Challenges and Opportunities](https://mlsanity.com/paper/2111.00364/) (2021)
- [A Constructive Prediction of the Generalization Error Across Scales](https://mlsanity.com/paper/1909.12673/) (2019)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [AI and Memory Wall](https://mlsanity.com/paper/2403.14123/) (2024)
- [Neural network representation of quantum systems](https://mlsanity.com/paper/2403.11420/) (2024)
- [Optically Connected Memory for Disaggregated Data Centers](https://mlsanity.com/paper/2008.10802/) (2020)
- [Low Latency Computing for Time Stretch Instruments](https://mlsanity.com/paper/2304.02249/) (2023)

## Abstract (arXiv)

Deep neural networks (DNNs) have achieved exceptional performance across various fields by learning complex, nonlinear mappings from large-scale datasets. However, they face challenges such as high memory requirements and computational costs with limited interpretability. This paper introduces an approach where master equations of physics are converted into multilayered networks that are trained via backpropagation. The resulting general-purpose model effectively encodes data in the properties of the underlying physical system. In contrast to existing methods wherein a trained neural network is used as a computationally efficient alternative for solving physical equations, our approach directly treats physics equations as trainable models. We demonstrate this physical embedding concept with the Nonlinear Schr"odinger Equation (NLSE), which acts as trainable architecture for learning complex patterns including nonlinear mappings and memory effects from data. The network embeds data representation in orders of magnitude fewer parameters than conventional neural networks when tested on time series data. Notably, the trained "Nonlinear Schr"odinger Network" is interpretable, with all parameters having physical meanings. This interpretability offers insight into the underlying dynamics of the system that produced the data. The proposed method of replacing traditional DNN feature learning architectures with physical equations is also extended to the Gross-Pitaevskii Equation, demonstrating the broad applicability of the framework to other master equations of physics. Among our results, an ablation study quantifies the relative importance of physical terms such as dispersion, nonlinearity, and potential energy for classification accuracy. We also outline the limitations of this approach as it relates to generalizability.