Training-Free Spectral Fingerprints of Voice Processing in Transformers
Not on the map. Why?
Abstract (arXiv)
Different transformer architectures implement identical linguistic computations via distinct connectivity patterns, yielding model imprinted ``computational fingerprints'' detectable through spectral analysis. Using graph signal processing on attention induced token graphs, we track changes in algebraic connectivity (Fiedler value, Deltaλ2) under voice alternation across 20 languages and three model families, with a prespecified early window (layers 2--5). Our analysis uncovers clear architectural signatures: Phi-3-Mini shows a dramatic English specific early layer disruption (overlineDeltaλ2[2,5]!≈!-0.446) while effects in 19 other languages are minimal, consistent with public documentation that positions the model primarily for English use. Qwen2.5-7B displays small, distributed shifts that are largest for morphologically rich languages, and LLaMA-3.2-1B exhibits systematic but muted responses. These spectral signatures correlate strongly with behavioral differences (Phi-3: r=-0.976) and are modulated by targeted attention head ablations, linking the effect to early attention structure and confirming functional relevance. Taken together, the findings are consistent with the view that training emphasis can leave detectable computational imprints: specialized processing strategies that manifest as measurable connectivity patterns during syntactic transformations. Beyond voice alternation, the framework differentiates reasoning modes, indicating utility as a simple, training free diagnostic for revealing architectural biases and supporting model reliability analysis.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 18 of its 32 references that are in the map. First posted on arXiv 21 Oct 2025.
Builds on
18 of 32 references.
References · 18
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,916
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,181
- 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,086
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,145
- 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,101
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,049
- 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 999
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 850
- 2020A Primer in BERTology: What we know about how BERT workscited by 809
- 2020Quantifying Attention Flow in Transformerscited by 767
6 more
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 711
- 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 557
- 2025Chain of Draft: Thinking Faster by Writing Lesscited by 292
- 2019Assessing BERT's Syntactic Abilitiescited by 191
- 2023Birth of a Transformer: A Memory Viewpointcited by 156
- 2025Metacognitive Reuse: Turning Recurring LLM Reasoning Into Concise Behaviorscited by 30
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.