Zero-Direction Probing: A Linear-Algebraic Framework for Deep Analysis of Large-Language-Model Drift
Not on the map. Why?
Abstract (arXiv)
We present Zero-Direction Probing (ZDP), a theory-only framework for detecting model drift from null directions of transformer activations without task labels or output evaluations. Under assumptions A1--A6, we prove: (i) the Variance--Leak Theorem, (ii) Fisher Null-Conservation, (iii) a Rank--Leak bound for low-rank updates, and (iv) a logarithmic-regret guarantee for online null-space trackers. We derive a Spectral Null-Leakage (SNL) metric with non-asymptotic tail bounds and a concentration inequality, yielding a-priori thresholds for drift under a Gaussian null model. These results show that monitoring right/left null spaces of layer activations and their Fisher geometry provides concrete, testable guarantees on representational change.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 21 references that are in the map. First posted on arXiv 9 Aug 2025.
Builds on
7 of 21 references.
References · 7
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
- 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 236
- 2021On the Variance of the Fisher Information for Deep Learningcited by 22
- 2025Put the Space of LoRA Initialization to the Extreme to Preserve Pre-trained Knowledgecited by 16
- 2025A Free Probabilistic Framework for Analyzing the Transformer-based Language Modelscited by 1
Built on by
Not cited yet.