mlsanityData to 7 Oct 2026

Support Operation Factorization: Compositional Readout of Frozen Vision Encoders under Controlled Interventions

arXiv 2608.06174 ↗First posted 6 Aug 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Compositional analysis of frozen vision encoders should determine both what changed and where it changed. Standard factor probes score these axes separately, however, and can reward multiple operations that reuse the same predicted slot. We call this failure operation laundering. We introduce an injectively aligned leave-one-cell-out protocol over support x operation grids and SO-OPF, a readout that factors cell energy into support salience and a competitive operation posterior. This formulation separates two questions that aggregate scores conflate: whether the carrier composes held-out bindings when the grid is known, and whether that grid can be recovered from flat cell labels. With frozen DINOv3 features, known factorial assignment reaches 0.874 injective accuracy on Shapes3D-Extended and 0.799 on globally image-disjoint COCO; learning the assignment from flat labels reaches 0.769 and 0.762, respectively. Under matched-axis-aware supervision on Shapes3D, the factored carrier improves learned-assignment accuracy from 0.653 to 0.841 over a dense carrier and eliminates its laundering gap. SigLIP2 replicates the COCO separation. A rebuilt MuJoCo substrate exposes a boundary: learned-assignment accuracy is 0.569 with DINOv3 and 0.484 with SigLIP2, with substantial slot collapse. Thus factored readout and injective evaluation recover held-out bindings on two substrates while exposing, rather than hiding, a renderer-specific failure boundary; they do not establish universal recovery from flat labels.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 34 references that are in the map. First posted on arXiv 6 Aug 2026.

Builds on

19 of 34 references.

References · 19

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  3. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  4. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  5. 2020Object-Centric Learning with Slot Attentioncited by 865
  6. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  7. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  8. 2018Disentangling by Factorisingcited by 614
  9. 2019Designing and Interpreting Probes with Control Taskscited by 608
  10. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  11. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  12. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
7 more
  1. 2018Towards a Definition of Disentangled Representationscited by 294
  2. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 280
  3. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  4. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  5. 2020On Disentangled Representations Learned From Correlated Datacited by 94
  6. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  7. 2026Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Modelscited by 11

Built on by

Not cited yet.