RSD: Moving Local Triangular Charts for Auditing Language-Model Hidden States
Not on the map. Why?
Abstract (arXiv)
We study Relational Semantic Decomposition, abbreviated as RSD, as a moving local triangular chart audit for language-model hidden states. For repeated occurrences of one target word, RSD fits a shared three-anchor membership chart St at layer or token-time t. The hidden-state channel uses Xt≈ StCt; the invariant readout Mt=StSt^top is the induced occurrence co-membership relation, and Rt=Xt-StCt records what the fitted root chart leaves outside the chart. The broader joint audit reuses the same membership chart for relation data, At≈ StBtSt^top, such as an attention-derived occurrence relation. The current GPT-2 evidence is the X-channel hidden-state audit with Word-in-Context labels used as an external same-sense versus different-sense reference relation. On full WiC train, the root chart passes 16 of 53 eligible target words; this is audit coverage, not GPT-2 task accuracy. Token-time and pair-level diagnostics show the main regimes: textttmake and textttbreak align at the target state, textttdrive and textttstay improve after right context in small-count exploratory cases, and textttplay remains a localized root-chart failure whose final same-sense pairs are not closer and have larger residual discrepancy. The resulting claim is diagnostic: RSD reports where a sense relation is visible in root co-membership and which failures become residual branch candidates or attention-channel obligations.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 17 references that are in the map. First posted on arXiv 17 May 2026.
Builds on
4 of 17 references.
References · 4
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
- 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
- 2026Symmetry in language statistics shapes the geometry of model representationscited by 24
Built on by
Not cited yet.