mlsanityData to 14 Oct 2026

Image-guided topic modeling for interpretable privacy classification

arXiv 2409.18674 ↗First posted 27 Sep 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Predicting and explaining the private information contained in an image in human-understandable terms is a complex and contextual task. This task is challenging even for large language models. To facilitate the understanding of privacy decisions, we propose to predict image privacy based on a set of natural language content descriptors. These content descriptors are associated with privacy scores that reflect how people perceive image content. We generate descriptors with our novel Image-guided Topic Modeling (ITM) approach. ITM leverages, via multimodality alignment, both vision information and image textual descriptions from a vision language model. We use the ITM-generated descriptors to learn a privacy predictor, Priv×ITM, whose decisions are interpretable by design. Our Priv×ITM classifier outperforms the reference interpretable method by 5 percentage points in accuracy and performs comparably to the current non-interpretable state-of-the-art model.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 29 of its 76 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

29 of 76 references.

References · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  10. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  11. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  12. 2021Grounded Language-Image Pre-trainingcited by 1,156
17 more
  1. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  2. 2020Concept Bottleneck Modelscited by 1,032
  3. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  4. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 312
  5. 2022Post-hoc Concept Bottleneck Modelscited by 293
  6. 2023Label-Free Concept Bottleneck Modelscited by 285
  7. 2020Parametric UMAP embeddings for representation and semi-supervised learningcited by 104
  8. 2017Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Imagescited by 101
  9. 2019Interpretability Beyond Classification Output: Semantic Bottleneck Networkscited by 49
  10. 2023Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Modelscited by 40
  11. 2022Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?cited by 31
  12. 2019Image Privacy Prediction Using Deep Neural Networkscited by 13
  13. 2020Zero-Bias Deep Learning for Accurate Identification of Internet of Things (IoT) Devicescited by 10
  14. 2019Dynamic Deep Multi-modal Fusion for Image Privacy Predictioncited by 7
  15. 2022DRAG: Dynamic Region-Aware GCN for Privacy-Leaking Image Detectioncited by 4
  16. 2022Content-based Graph Privacy Advisorcited by 4
  17. 2023Human-interpretable and deep features for image privacy classificationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025On the Robustness of Vision-Language Models in Zero-shot Privacy Classificationcited by 1
  2. 2025The impact of abstract and object tags on image privacy classificationcited by 0
  3. 2025Cross-modal Counterfactual Explanations: Uncovering Decision Factors and Dataset Biases in Subjective Classificationcited by 0
  4. 2026PrivLEX: Detecting legal concepts in images through Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Assessing Visual Privacy Risks in Multimodal AI: A Novel Taxonomy-Grounded Evaluation of Vision-Language Models
  2. 2024Private Attribute Inference from Images with Vision-Language Models
  3. 2025MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models
  4. 2026Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs
  5. 2025The Eye of Sherlock Holmes: Uncovering User Private Attribute Profiling via Vision-Language Model Agentic Framework
  6. 2025Through Their Eyes: User Perceptions on Sensitive Attribute Inference of Social Media Videos by Visual Language Models
  7. 2025Beyond Blanket Masking: Examining Granularity for Privacy Protection in Images Captured by Blind and Low Vision Users
  8. 2024Privacy-Aware Visual Language Models