mlsanityData to 14 Oct 2026

KD-PINN: Knowledge-Distilled PINNs for ultra-low-latency real-time neural PDE solvers

arXiv 2512.13336 ↗First posted 15 Dec 2025cs.LG cs.NA math.NAcited by 0

Not on the map. Why?

Abstract (arXiv)

This work introduces Knowledge-Distilled Physics-Informed Neural Networks (KD-PINN), a framework that transfers the predictive accuracy of a high-capacity teacher model to a compact student through a continuous adaptation of the Kullback-Leibler divergence. In order to confirm its generality for various dynamics and dimensionalities, the framework is evaluated on a representative set of partial differential equations (PDEs). Across the considered benchmarks, the student model achieves inference speedups ranging from x4.8 (Navier-Stokes) to x6.9 (Burgers), while preserving accuracy. Accuracy is improved by on the order of 1% when the model is properly tuned. The distillation process also revealed a regularizing effect. With an average inference latency of 5.3 ms on CPU, the distilled models enter the ultra-low-latency real-time regime defined by sub-10 ms performance. Finally, this study examines how knowledge distillation reduces inference latency in PINNs, to contribute to the development of accurate ultra-low-latency neural PDE solvers.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 45 references that are in the map. First posted on arXiv 15 Dec 2025.

Builds on

19 of 45 references.

References · 19

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  3. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  4. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  5. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  6. 2018On the Spectral Bias of Neural Networkscited by 1,341
  7. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  8. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  9. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
  10. 2020What is the State of Neural Network Pruning?cited by 583
  11. 2019Variational Information Distillation for Knowledge Transfercited by 324
  12. 2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEscited by 116
7 more
  1. 2022Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systemscited by 54
  2. 2021Meta-learning PINN loss functionscited by 51
  3. 2022Adaptive Self-supervision Algorithms for Physics-informed Neural Networkscited by 24
  4. 2018Ultra-Low Latency (ULL) Networks: The IEEE TSN and IETF DetNet Standards and Related 5G ULL Researchcited by 7
  5. 2015Realizing the Tactile Internet: Haptic Communications over Next Generation 5G Cellular Networkscited by 4
  6. 2022An Experimental Evaluation of Machine Learning Training on a Real Processing-in-Memory Systemcited by 2
  7. 2022Deep Learning Inference Frameworks Benchmarkcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Sub-Sequential Physics-Informed Learning with State Space Model
  2. 2023LatentPINNs: Generative physics-informed neural networks via a latent representation learning
  3. 2025Tensor-Compressed and Fully-Quantized Training of Neural PDE Solvers
  4. 2026Oscillatory State-Space Models as Inductive Biases for Physics-Informed Neural PDE Solvers
  5. 2025Lang-PINN: From Language to Physics-Informed Neural Networks via a Multi-Agent Framework
  6. 2025S2GPT-PINNs: Sparse and Small models for PDEs
  7. 2025PINN Balls: Scaling Second-Order Methods for PINNs with Domain Decomposition and Adaptive Sampling
  8. 2024Physics Informed Distillation for Diffusion Models