# A Graph-Based Inspection and Intervention Tool for Assessing Mechanistic Learning in PINNs

arXiv: https://arxiv.org/abs/2610.04939 · HTML: https://mlsanity.com/paper/2610.04939/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 13 references that are in the map.
- First posted on arXiv 4 Oct 2026.

## Builds on (most cited first, 8 of 8)

- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability](https://mlsanity.com/paper/2301.04709/) (2023)
- [Explainable Machine Learning for Scientific Insights and Discoveries](https://mlsanity.com/paper/1905.08883/) (2019)
- [Understanding the Difficulty of Solving Cauchy Problems with PINNs](https://mlsanity.com/paper/2405.02561/) (2024)
- [Causal Dependence Plots](https://mlsanity.com/paper/2303.04209/) (2023)
- [PWSHAP: A Path-Wise Explanation Model for Targeted Variables](https://mlsanity.com/paper/2306.14672/) (2023)
- [REX: Causal discovery based on machine learning and explainability techniques](https://mlsanity.com/paper/2501.12706/) (2025)

## Abstract (arXiv)

We ask whether physically meaningful correspondences discovered inside a trained scientific model remain meaningful outside the conditions under which they were discovered. We introduce GIIT (Graph-based Inspection and Intervention Tool), which represents governing physics as a computational physics dependency graph, maps graph nodes to internal network components via sensitivity- and trend-based discovery, and tests the resulting mapping under targeted intervention. Evaluating on temporal extrapolation out-of-distribution (OOD) regimes of increasing severity, we find that temporal extrapolation is associated with layer-wise correspondence drift and reduced functional correspondence. Specifically, while the model achieves low physics residual in-distribution (1.915 x 10-4 on full ID and 1.01 x 10-4 on an ID sub-window t in [0.5, 0.8]), the functional mapping changes even before leaving the training domain, and the shift continues as the evaluation window extends beyond the training domain: residual error increases from 2.81 x 10-2 on the boundary-crossing window (t in [0.5, 1.5]) to 3.30 x 10-1 on severe OOD (t in [1, 2]), accompanied by a systematic leftward shift of internal layer mappings, where the average winner layer index drops from 5.71 (ID) to 3.00 (ID sub-window) down to 2.29 (severe OOD). Only 2 of 7 physical nodes maintain stable layer assignments across the boundary-crossing window, and only 1 of 7 under severe extrapolation, revealing potential internal functional correspondence changes before severe degradation in conventional output metrics. Additional results for linear oscillatory systems are further detailed in the appendix.