mlsanityData to 14 Oct 2026

Correlates of Image Memorability in Vision Encoders: Activations, Attention Entropy, Patch Uniformity and Autoencoder Losses

arXiv 2509.01453 ↗First posted 1 Sep 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Images vary in how memorable they are to humans. Inspired by findings from cognitive science and computer vision, we explore correlates of image memorability in pretrained transformer-based vision encoders for the first time. Focusing initially on activations, attention distributions, and the uniformity of image patches, we find that these features correlate with memorability to some extent. Additionally, we explore sparse autoencoder loss over the representations of vision encoders as a proxy for memorability, which yields results outperforming past methods using convolutional neural network representations. Our results shed light on the relationship between model-internal features and memorability. They show that some features are informative predictors of what makes images memorable to humans; revealing that, in particular, the reconstruction loss from our autoencoders is a strong correlate of image memorability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 42 references that are in the map. First posted on arXiv 1 Sep 2025.

Builds on

18 of 42 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  6. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  7. 2022Toy Models of Superpositioncited by 971
  8. 2023Vision Transformers Need Registerscited by 871
  9. 2024Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)cited by 119
  10. 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 113
  11. 2025Interpreting the linear structure of vision-language model embedding spacescited by 32
  12. 2025Interpretable and Testable Vision Features via Sparse Autoencoderscited by 25
6 more
  1. 2021Embracing New Techniques in Deep Learning for Estimating Image Memorabilitycited by 10
  2. 2025Probing the Representational Power of Sparse Autoencoders in Vision Modelscited by 10
  3. 2023Image Memorability Prediction with Vision Transformerscited by 9
  4. 2021Memorability: An image-computable measure of information utilitycited by 8
  5. 2024Modeling Visual Memorability Assessment with Autoencoders Reveals Characteristics of Memorable Imagescited by 2
  6. 2022Comprehensive Literature Survey on Deep Learning used in Image Memorability Prediction and Modificationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Unforgettable Lessons from Forgettable Images: Intra-Class Memorability Matters in Computer Vision
  2. 2025Unsupervised Memorability Modeling from Tip-of-the-Tongue Retrieval Queries
  3. 2023What do larger image classifiers memorise?
  4. 2026How to Take a Memorable Picture? Empowering Users with Actionable Feedback
  5. 2024D'ej`a Vu Memorization in Vision-Language Models
  6. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models
  7. 2026Lost or Hidden? A Concept-Level Forgetting in Supervised Continual Learning
  8. 2025Quantifying Cross-Modality Memorization in Vision-Language Models