# Tensor-Compressed and Fully-Quantized Training of Neural PDE Solvers

arXiv: https://arxiv.org/abs/2512.09202 · HTML: https://mlsanity.com/paper/2512.09202/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 10 Dec 2025.

## Builds on (most cited first, 19 of 19)

- [Microscaling Data Formats for Deep Learning](https://mlsanity.com/paper/2310.10537/) (2023)
- [SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training](https://mlsanity.com/paper/2505.11594/) (2025)
- [With Shared Microexponents, A Little Shifting Goes a Long Way](https://mlsanity.com/paper/2302.08007/) (2023)
- [Physics-Informed Neural Nets for Control of Dynamical Systems](https://mlsanity.com/paper/2104.02556/) (2021)
- [Learning Physics-Informed Neural Networks without Stacked Back-propagation](https://mlsanity.com/paper/2202.09340/) (2022)
- [Hutchinson Trace Estimation for High-Dimensional and High-Order Physics-Informed Neural Networks](https://mlsanity.com/paper/2312.14499/) (2023)
- [Functional Tensor Decompositions for Physics-Informed Neural Networks](https://mlsanity.com/paper/2408.13101/) (2024)
- [Guaranteed Approximation Bounds for Mixed-Precision Neural Operators](https://mlsanity.com/paper/2307.15034/) (2023)
- [Speeding up and reducing memory usage for scientific machine learning via mixed precision](https://mlsanity.com/paper/2401.16645/) (2024)
- [DaCapo: Accelerating Continuous Learning in Autonomous Systems for Video Analytics](https://mlsanity.com/paper/2403.14353/) (2024)
- [TT-PINN: A Tensor-Compressed Neural PDE Solver for Edge Computing](https://mlsanity.com/paper/2207.01751/) (2022)
- [Physics-informed neural networks for non-Newtonian fluid thermo-mechanical problems: an application to rubber calendering process](https://mlsanity.com/paper/2201.13389/) (2022)
- [Efficient Neural PDE-Solvers using Quantization Aware Training](https://mlsanity.com/paper/2308.07350/) (2023)
- [Physics-Informed Neural Networks for Dynamic Process Operations with Limited Physical Knowledge and Data](https://mlsanity.com/paper/2406.01528/) (2024)
- [Efficient Precision-Scalable Hardware for Microscaling (MX) Processing in Robotics Learning](https://mlsanity.com/paper/2505.22404/) (2025)
- [Real-Time FJ/MAC PDE Solvers via Tensorized, Back-Propagation-Free Optical PINN Training](https://mlsanity.com/paper/2401.00413/) (2023)
- [Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization](https://mlsanity.com/paper/2501.06663/) (2025)
- [Scalable Back-Propagation-Free Training of Optical Physics-Informed Neural Networks](https://mlsanity.com/paper/2502.12384/) (2025)
- [FETTA: Flexible and Efficient Hardware Accelerator for Tensorized Neural Network Training](https://mlsanity.com/paper/2504.06474/) (2025)

## Built on by (most cited first, 1 of 1)

- [Operator Boosting Produces Pareto-Efficient PDE Surrogates](https://mlsanity.com/paper/2606.17460/) (2026)

## Abstract (arXiv)

Physics-Informed Neural Networks (PINNs) have emerged as a promising paradigm for solving partial differential equations (PDEs) by embedding physical laws into neural network training objectives. However, their deployment on resource-constrained platforms is hindered by substantial computational and memory overhead, primarily stemming from higher-order automatic differentiation, intensive tensor operations, and reliance on full-precision arithmetic. To address these challenges, we present a framework that enables scalable and energy-efficient PINN training on edge devices. This framework integrates fully quantized training, Stein's estimator (SE)-based residual loss computation, and tensor-train (TT) decomposition for weight compression. It contributes three key innovations: (1) a mixed-precision training method that use a square-block MX (SMX) format to eliminate data duplication during backpropagation; (2) a difference-based quantization scheme for the Stein's estimator that mitigates underflow; and (3) a partial-reconstruction scheme (PRS) for TT-Layers that reduces quantization-error accumulation. We further design PINTA, a precision-scalable hardware accelerator, to fully exploit the performance of the framework. Experiments on the 2-D Poisson, 20-D Hamilton-Jacobi-Bellman (HJB), and 100-D Heat equations demonstrate that the proposed framework achieves accuracy comparable to or better than full-precision, uncompressed baselines while delivering 5.5x to 83.5x speedups and 159.6x to 2324.1x energy savings. This work enables real-time PDE solving on edge devices and paves the way for energy-efficient scientific computing at scale.