mlsanityData to 7 Oct 2026

PIXELMOD: Improving Soft Moderation of Visual Misleading Information on Twitter

arXiv 2407.20987 ↗First posted 30 Jul 2024cs.CV cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

Images are a powerful and immediate vehicle to carry misleading or outright false messages, yet identifying image-based misinformation at scale poses unique challenges. In this paper, we present PIXELMOD, a system that leverages perceptual hashes, vector databases, and optical character recognition (OCR) to efficiently identify images that are candidates to receive soft moderation labels on Twitter. We show that PIXELMOD outperforms existing image similarity approaches when applied to soft moderation, with negligible performance overhead. We then test PIXELMOD on a dataset of tweets surrounding the 2020 US Presidential Election, and find that it is able to identify visually misleading images that are candidates for soft moderation with 0.99% false detection and 2.06% false negatives.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 86 references that are in the map. First posted on arXiv 30 Jul 2024.

Builds on

28 of 86 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2017Billion-scale similarity search with GPUscited by 2,477
  8. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  9. 2018Learning Rich Features for Image Manipulation Detectioncited by 154
  10. 2021Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resourcescited by 71
  11. 2018On the Origins of Memes by Means of Fringe Web Communitiescited by 45
  12. 2022BadHash: Invisible Backdoor Attacks against Deep Hashing with Clean Labelcited by 28
16 more
  1. 2021COSMOS: Catching Out-of-Context Misinformation with Self-Supervised Learningcited by 26
  2. 2022On the Evolution of (Hateful) Memes by Means of Multimodal Contrastive Learningcited by 26
  3. 2022Understanding and Detecting Hateful Content using Contrastive Learningcited by 13
  4. 2020Images and Misinformation in Political Groups: Evidence from WhatsApp in Indiacited by 11
  5. 2021"I Won the Election!": An Empirical Analysis of Soft Moderation Interventions on Twittercited by 10
  6. 2021TROLLMAGNIFIER: Detecting State-Sponsored Troll Accounts on Redditcited by 10
  7. 2020Understanding the Use of Fauxtography on Social Mediacited by 9
  8. 2022LAMBRETTA: Learning to Rank for Twitter Soft Moderationcited by 8
  9. 2017Hiding in Plain Sight: A Longitudinal Study of Combosquatting Abusecited by 7
  10. 2021VoterFraud2020: a Multi-modal Dataset of Election Fraud Claims on Twittercited by 7
  11. 2016Dial One for Scam: A Large-Scale Analysis of Technical Support Scamscited by 6
  12. 2022Toxicity in the Decentralized Web and the Potential for Model Sharingcited by 6
  13. 2021Identifying Misinformation from Website Screenshotscited by 5
  14. 2021Tiplines to Combat Misinformation on Encrypted Platforms: A Case Study of the 2019 Indian Election on WhatsAppcited by 2
  15. 2022Coordinated through aWeb of Images: Analysis of Image-based Influence Operations from China, Iran, Russia, and Venezuelacited by 2
  16. 2020A First Look at COVID-19 Messages on WhatsApp in Pakistancited by 1

Built on by

Not cited yet.