# Iterative Erasure Count Is Not an Affine-Invariant Concept Dimension

arXiv: https://arxiv.org/abs/2608.10566 · HTML: https://mlsanity.com/paper/2608.10566/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 30 references that are in the map.
- First posted on arXiv 11 Aug 2026.

## Builds on (most cited first, 25 of 25)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Masked Feature Prediction for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2112.09133/) (2021)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction](https://mlsanity.com/paper/2203.01577/) (2022)
- [Understanding Human Hands in Contact at Internet Scale](https://mlsanity.com/paper/2006.06669/) (2020)
- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [Information-Theoretic Probing for Linguistic Structure](https://mlsanity.com/paper/2004.03061/) (2020)
- [On Linear Identifiability of Learned Representations](https://mlsanity.com/paper/2007.00810/) (2020)
- [Linear Adversarial Concept Erasure](https://mlsanity.com/paper/2201.12091/) (2022)
- [TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding](https://mlsanity.com/paper/2401.08399/) (2024)
- [Gold Doesn't Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Information](https://mlsanity.com/paper/2203.07893/) (2022)
- [StillFast: An End-to-End Approach for Short-Term Object Interaction Anticipation](https://mlsanity.com/paper/2304.03959/) (2023)
- [Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection](https://mlsanity.com/paper/2212.04273/) (2022)
- [EgoPCA: A New Framework for Egocentric Hand-Object Interaction Understanding](https://mlsanity.com/paper/2309.02423/) (2023)
- [Interventional Probing in High Dimensions: An NLI Case Study](https://mlsanity.com/paper/2304.10346/) (2023)
- [Improving Causal Interventions in Amnesic Probing with Mean Projection or LEACE](https://mlsanity.com/paper/2506.11673/) (2025)
- [Detecting Precise Hand Touch Moments in Egocentric Video](https://mlsanity.com/paper/2604.12343/) (2026)
- [Gauge Freedom and Metric Dependence in Neural Representation Spaces](https://mlsanity.com/paper/2603.06774/) (2026)

## Built on by (most cited first, 1 of 1)

- [Detecting a Shift Is Not Enough: Exact Minimax Limits of Linear Representation Repair](https://mlsanity.com/paper/2610.08069/) (2026)

## Abstract (arXiv)

How many directions does a neural representation use to encode a concept? A common answer repeatedly erases probe directions and reports the stopping count or cumulative removed rank. We show that both quantities can change under an information-preserving invertible reparameterization, so neither is intrinsically a concept dimension. We distinguish model-defined population quantities (generating dimension, sufficient linear dimension, and minimum guarding rank) from procedure-defined quantities such as stopping count and cumulative edit rank. In a population Gaussian construction, an invertible shear preserves the prediction problem and all three quantities, yet changes the cumulative Euclidean erasure count from one to two. The separation holds for Moore--Penrose ordinary least squares and every finite nonnegative ridge weight. For a two-output full-QR procedure matching our motivating video analysis, cumulative edit rank similarly changes from two to the ambient dimension four. Conversely, the complete cumulative metric-QR trajectory is affine-equivariant when its positive-definite metric, probe, regularizer, and tie-breaking are transported consistently; exact covariance is one corollary, not a canonical semantic metric. In a known-rank finite-sample Adam/QR calibration, identity mixing stops after one accepted update in all 20 large-sample runs, whereas each tested shear a∈.5,.75,1,1.25,2 accepts at least two updates in all 20 runs. Controlled reparameterizations of frozen V-JEPA2 features preserve rank-zero predictions yet alter later Euclidean trajectories under practical optimization. These visual contact experiments are stress tests, not estimates of contact dimension. Iterative erasure therefore returns a procedure-relative estimand jointly determined by representation geometry and the full measurement procedure, not a semantic dimension by itself.