mlsanityData to 7 Oct 2026

Textual Training for the Hassle-Free Removal of Unwanted Visual Data: Case Studies on OOD and Hateful Image Detection

arXiv 2409.19840 ↗First posted 30 Sep 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In our study, we explore methods for detecting unwanted content lurking in visual datasets. We provide a theoretical analysis demonstrating that a model capable of successfully partitioning visual data can be obtained using only textual data. Based on the analysis, we propose Hassle-Free Textual Training (HFTT), a streamlined method capable of acquiring detectors for unwanted visual content, using only synthetic textual data in conjunction with pre-trained vision-language models. HFTT features an innovative objective function that significantly reduces the necessity for human involvement in data annotation. Furthermore, HFTT employs a clever textual data synthesis method, effectively emulating the integration of unknown visual data distribution into the training process at no extra cost. The unique characteristics of HFTT extend its utility beyond traditional out-of-distribution detection, making it applicable to tasks that address more abstract concepts. We complement our analyses with experiments in out-of-distribution detection and hateful image detection. Our codes are available at https://github.com/Saehyung-Lee/HFTT

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 51 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

42 of 51 references.

References · 42

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  10. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  11. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  12. 2013Describing Textures in the Wildcited by 2,110
30 more
  1. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  2. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  3. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  4. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  5. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  6. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  7. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 657
  8. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  9. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  10. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  11. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  12. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  13. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  14. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  15. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  16. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  17. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  18. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  19. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  20. 2021Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIPcited by 151
  21. 2023In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluationcited by 132
  22. 2023CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say Nocited by 130
  23. 2023Non-Parametric Outlier Synthesiscited by 129
  24. 2023DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Trainingcited by 84
  25. 2024Negative Label Guided OOD Detection with Pretrained Vision-Language Modelscited by 66
  26. 2023Into the LAIONs Den: Investigating Hate in Multimodal Datasetscited by 54
  27. 2022Texts as Images in Prompt Tuning for Multi-Label Image Recognitioncited by 51
  28. 2023LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collectionscited by 43
  29. 2022Understanding and Detecting Hateful Content using Contrastive Learningcited by 13
  30. 2023On the Powerfulness of Textual Outlier Exposure for Visual OoD Detectioncited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023CiT: Curation in Training for Effective Vision-Language Data
  2. 2023Generating Enhanced Negatives for Training Language-Based Object Detectors
  3. 2024Clean Evaluations on Contaminated Visual Language Models
  4. 2024From Training-Free to Adaptive: Empirical Insights into MLLMs' Understanding of Detection Information
  5. 2026TextRich: A Multi-Domain Benchmark for Detecting AI-Generated Text-Rich Images from GPT-Image-2
  6. 2025Cross-Modal Transfer from Memes to Videos: Addressing Data Scarcity in Hateful Video Detection
  7. 2026Training-Free and Interpretable Hateful Video Detection via Multi-stage Adversarial Reasoning
  8. 2025Language-Assisted Feature Transformation for Anomaly Detection