# Zero-Ablation Overstates Register Content Dependence in DINO Vision Transformers

arXiv: https://arxiv.org/abs/2604.14433 · HTML: https://mlsanity.com/paper/2604.14433/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Builds on (most cited first, 28 of 28)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [SPair-71k: A Large-scale Benchmark for Semantic Correspondence](https://mlsanity.com/paper/1908.10543/) (2019)
- [What Do Self-Supervised Vision Transformers Learn?](https://mlsanity.com/paper/2305.00729/) (2023)
- [The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations](https://mlsanity.com/paper/2106.00786/) (2021)
- [Vision Transformers Don't Need Trained Registers](https://mlsanity.com/paper/2506.08010/) (2025)
- [Denoising Vision Transformers](https://mlsanity.com/paper/2401.02957/) (2024)
- [Into the Rabbit Hull: From Task-Relevant Concepts in DINO to Minkowski Geometry](https://mlsanity.com/paper/2510.08638/) (2025)
- [SINDER: Repairing the Singular Defects of DINOv2](https://mlsanity.com/paper/2407.16826/) (2024)
- [Near, far: Patch-ordering enhances vision foundation models' scene understanding](https://mlsanity.com/paper/2408.11054/) (2024)
- [Vision Transformers Need More Than Registers](https://mlsanity.com/paper/2602.22394/) (2026)
- [Vision Transformers with Self-Distilled Registers](https://mlsanity.com/paper/2505.21501/) (2025)
- [Leveraging Registers in Vision Transformers for Robust Adaptation](https://mlsanity.com/paper/2501.04784/) (2025)
- [Grounding Intelligence in Movement](https://mlsanity.com/paper/2507.02771/) (2025)

## Abstract (arXiv)

Zero-ablation -- replacing token activations with zero vectors -- is widely used to probe token function in vision transformers. Register zeroing in DINOv2+registers and DINOv3 produces large drops (up to -36.6,pp classification, -30.9,pp segmentation), suggesting registers are functionally indispensable. However, three replacement controls -- mean-substitution, noise-substitution, and cross-image register-shuffling -- preserve performance across classification, correspondence, and segmentation, remaining within ~1,pp of the unmodified baseline. Per-patch cosine similarity shows these replacements genuinely perturb internal representations, while zeroing causes disproportionately large perturbations, consistent with why it alone degrades tasks. We conclude that zero-ablation overstates dependence on exact register content. In the frozen-feature evaluations we test, performance depends on plausible register-like activations rather than on exact image-specific values. Registers nevertheless buffer dense features from texttt[CLS] dependence and are associated with compressed patch geometry. These findings, including the replacement-control results, replicate at ViT-B scale.