mlsanityData to 14 Oct 2026

Using a Cross-Task Grid of Linear Probes to Interpret CNN Model Predictions On Retinal Images

arXiv 2107.11468 ↗First posted 23 Jul 2021cs.LG cs.CV eess.IVcited by 0

On the map: Mechanistic interpretability → Interpreting vision and multimodal models

Abstract (arXiv)

We analyze a dataset of retinal images using linear probes: linear regression models trained on some "target" task, using embeddings from a deep convolutional (CNN) model trained on some "source" task as input. We use this method across all possible pairings of 93 tasks in the UK Biobank dataset of retinal images, leading to ~164k different models. We analyze the performance of these linear probes by source and target task and by layer depth. We observe that representations from the middle layers of the network are more generalizable. We find that some target tasks are easily predicted irrespective of the source task, and that some other target tasks are more accurately predicted from correlated source tasks than from embeddings trained on the same task.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpreting vision and multimodal models, one of 13 directions in Mechanistic interpretability. Builds on 8 of its 19 references that are in the map. First posted on arXiv 23 Jul 2021.

Builds on

8 of 19 references.

References · 8

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  3. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  4. 2019Designing and Interpreting Probes with Control Taskscited by 640
  5. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  6. 2017Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learningcited by 63
  7. 2017Deep learning for predicting refractive error from retinal fundus imagescited by 5
  8. 2020Internal representation dynamics and geometry in recurrent neural networkscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Reverse Knowledge Distillation: Training a Large Model using a Small One for Retinal Image Matching on Limited Data
  2. 2026FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMs
  3. 2022INSightR-Net: Interpretable Neural Network for Regression using Similarity-based Comparisons to Prototypical Examples
  4. 2026Benchmarking Convolutional, Transformer, Hybrid, and Vision Language Models for Multi Disease Retinal Screening
  5. 2026Evaluation of Clinically Steerable Retinal Image Generation from Foundation Model Latent Spaces
  6. 2025Interpretable Few-Shot Retinal Disease Diagnosis with Concept-Guided Prompting of Vision-Language Models
  7. 2023A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision
  8. 2024Towards flexible perception with visual memory