# TAEN: A Model-Constrained Tikhonov Autoencoder Network for Forward and Inverse Problems

arXiv: https://arxiv.org/abs/2412.07010 · HTML: https://mlsanity.com/paper/2412.07010/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 72 references that are in the map.
- First posted on arXiv 9 Dec 2024.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [MoDL: Model Based Deep Learning Architecture for Inverse Problems](https://mlsanity.com/paper/1712.02862/) (2017)
- [The Little Engine that Could: Regularization by Denoising (RED)](https://mlsanity.com/paper/1611.02862/) (2016)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients](https://mlsanity.com/paper/1711.09404/) (2017)
- [Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems](https://mlsanity.com/paper/2111.02801/) (2021)
- [Transformers for Modeling Physical Systems](https://mlsanity.com/paper/2010.03957/) (2020)
- [Adversarial Regularizers in Inverse Problems](https://mlsanity.com/paper/1805.11572/) (2018)
- [NETT: Solving Inverse Problems with Deep Neural Networks](https://mlsanity.com/paper/1803.00092/) (2018)
- [Physics-informed neural networks for inverse problems in supersonic flows](https://mlsanity.com/paper/2202.11821/) (2022)
- [Medical image denoising using convolutional denoising autoencoders](https://mlsanity.com/paper/1608.04667/) (2016)
- [Scaleable input gradient regularization for adversarial robustness](https://mlsanity.com/paper/1905.11468/) (2019)
- [Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamics](https://mlsanity.com/paper/1912.11114/) (2019)
- [Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning](https://mlsanity.com/paper/2206.10745/) (2022)
- [A graph convolutional autoencoder approach to model order reduction for parametrized PDEs](https://mlsanity.com/paper/2305.08573/) (2023)
- [Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs](https://mlsanity.com/paper/2011.15110/) (2020)
- [Solving inverse-PDE problems with physics-aware neural networks](https://mlsanity.com/paper/2001.03608/) (2020)
- [Implicit Full Waveform Inversion with Deep Neural Representation](https://mlsanity.com/paper/2209.03525/) (2022)
- [Multi-Step Prediction of Dynamic Systems with Recurrent Neural Networks](https://mlsanity.com/paper/1806.00526/) (2018)
- [Machine Learning-augmented Predictive Modeling of Turbulent Separated Flows over Airfoils](https://mlsanity.com/paper/1608.03990/) (2016)
- [Long-time predictive modeling of nonlinear dynamical systems using neural networks](https://mlsanity.com/paper/1805.12547/) (2018)

## Built on by (most cited first, 3 of 3)

- [Good Things Come in Pairs: Paired Autoencoders for Inverse Problems](https://mlsanity.com/paper/2505.06549/) (2025)
- [Enhanced uncertainty quantification variational autoencoders for the solution of Bayesian inverse problems](https://mlsanity.com/paper/2502.13105/) (2025)
- [Physics-informed reservoir characterization from bulk and extreme pressure events with a differentiable simulator](https://mlsanity.com/paper/2604.13291/) (2026)

## Abstract (arXiv)

Efficient real-time solvers for forward and inverse problems are essential in engineering and science applications. Machine learning surrogate models have emerged as promising alternatives to traditional methods, offering substantially reduced computational time. Nevertheless, these models typically demand extensive training datasets to achieve robust generalization across diverse scenarios. While physics-based approaches can partially mitigate this data dependency and ensure physics-interpretable solutions, addressing scarce data regimes remains a challenge. Both purely data-driven and physics-based machine learning approaches demonstrate severe overfitting issues when trained with insufficient data. We propose a novel Tikhonov autoencoder model-constrained framework, called TAE, capable of learning both forward and inverse surrogate models using a single arbitrary observation sample. We develop comprehensive theoretical foundations including forward and inverse inference error bounds for the proposed approach for linear cases. For comparative analysis, we derive equivalent formulations for pure data-driven and model-constrained approach counterparts. At the heart of our approach is a data randomization strategy, which functions as a generative mechanism for exploring the training data space, enabling effective training of both forward and inverse surrogate models from a single observation, while regularizing the learning process. We validate our approach through extensive numerical experiments on two challenging inverse problems: 2D heat conductivity inversion and initial condition reconstruction for time-dependent 2D Navier-Stokes equations. Results demonstrate that TAE achieves accuracy comparable to traditional Tikhonov solvers and numerical forward solvers for both inverse and forward problems, respectively, while delivering orders of magnitude computational speedups.