mlsanityData to 14 Oct 2026

Mechanistic Interpretability of Fine-Tuned Vision Transformers on Distorted Images: Decoding Attention Head Behavior for Transparent and Trustworthy AI

arXiv 2503.18762 ↗First posted 24 Mar 2025cs.LG cs.AIcited by 5

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

Mechanistic interpretability improves the safety, reliability, and robustness of large AI models. This study examined individual attention heads in vision transformers (ViTs) fine tuned on distorted 2D spectrogram images containing non relevant content (axis labels, titles, color bars). By introducing extraneous features, the study analyzed how transformer components processed unrelated information, using mechanistic interpretability to debug issues and reveal insights into transformer architectures. Attention maps assessed head contributions across layers. Heads in early layers (1 to 3) showed minimal task impact with ablation increased MSE loss slightly (μ=0.11%, σ=0.09%), indicating focus on less critical low level features. In contrast, deeper heads (e.g., layer 6) caused a threefold higher loss increase (μ=0.34%, σ=0.02%), demonstrating greater task importance. Intermediate layers (6 to 11) exhibited monosemantic behavior, attending exclusively to chirp regions. Some early heads (1 to 4) were monosemantic but non task relevant (e.g. text detectors, edge or corner detectors). Attention maps distinguished monosemantic heads (precise chirp localization) from polysemantic heads (multiple irrelevant regions). These findings revealed functional specialization in ViTs, showing how heads processed relevant vs. extraneous information. By decomposing transformers into interpretable components, this work enhanced model understanding, identified vulnerabilities, and advanced safer, more transparent AI.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 6 of its 9 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

6 of 9 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention17 Mar

Other references · 5

  1. 2024How to use and interpret activation patchingcited by 214
  2. 2024From Explainable to Interpretable Deep Learning for Natural Language Processing in Healthcare: How Far from Reality?cited by 11
  3. 2025Interpretable Neural Causal Models with TRAM-DAGscited by 4
  4. 2025Towards Explainable Privacy Preservation in Federated Learning via Shapley Value-Guided Noise Injectioncited by 1
  5. 2025A Dual-Directional Context-Aware Test-Time Learning for Text Classificationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  2. 2026Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Interventioncited by 2
  3. 2025Vision Transformers Exhibit Human-Like Biases: Evidence of Orientation and Color Selectivity, Categorical Perception, and Phase Transitionscited by 0
  4. 2025DARTS-GT: Differentiable Architecture Search for Graph Transformers with Quantifiable Instance-Specific Interpretability Analysiscited by 0
  5. 2026Sparse but not Simpler: A Multi-Level Interpretability Analysis of Vision Transformerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Interpreting Transformers Through Attention Head Intervention
  2. 2023Scale Alone Does not Improve Mechanistic Interpretability in Vision Models
  3. 2026Capability neq Interpretability: Human Interpretability of Vision Foundation Models
  4. 2026Mechanistically Interpretable Neural Encoding Reveals Fine-Grained Functional Selectivity in Human Visual Cortex
  5. 2026Seeing Through Circuits: Faithful Mechanistic Interpretability for Vision Transformers
  6. 2026Cognitive Alignment At No Cost: Inducing Human Attention Biases For Interpretable Vision Transformers
  7. 2026Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning
  8. 2026Inside the Visual Mind: Neuroscience-Motivated Concept Circuits for Interpreting and Steering Vision Transformers