mlsanityData to 7 Oct 2026

Exploring Visual Engagement Signals for Representation Learning

arXiv 2104.07767 ↗First posted 15 Apr 2021cs.CV cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Visual engagement in social media platforms comprises interactions with photo posts including comments, shares, and likes. In this paper, we leverage such visual engagement clues as supervisory signals for representation learning. However, learning from engagement signals is non-trivial as it is not clear how to bridge the gap between low-level visual information and high-level social interactions. We present VisE, a weakly supervised learning approach, which maps social images to pseudo labels derived by clustered engagement signals. We then study how models trained in this way benefit subjective downstream computer vision tasks such as emotion recognition or political bias detection. Through extensive studies, we empirically demonstrate the effectiveness of VisE across a diverse set of classification tasks beyond the scope of conventional recognition.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 63 of its 93 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

63 of 93 references.

References · 63

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  12. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
38 more
  1. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  2. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  3. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  4. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  6. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  7. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  8. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  9. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  10. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  11. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  12. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  13. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  14. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  15. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  16. 2016Colorful Image Colorizationcited by 1,113
  17. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  18. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  19. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  20. 2017Focal Loss for Dense Object Detectioncited by 749
  21. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  22. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  23. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  24. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  25. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  26. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  27. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  28. 2019Large Scale Adversarial Representation Learningcited by 238
  29. 2015YFCC100M: The New Data in Multimedia Researchcited by 234
  30. 2019Billion-scale semi-supervised learning for image classificationcited by 232
  31. 2019Unsupervised Embedding Learning via Invariant and Spreading Instance Featurecited by 231
  32. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  33. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  34. 2019Scaling and Benchmarking Self-Supervised Visual Representation Learningcited by 201
  35. 2019Visual Semantic Reasoning for Image-Text Matchingcited by 185
  36. 2015Learning Visual Features from Large Weakly Supervised Datacited by 165
  37. 2020In Defense of Grid Features for Visual Question Answeringcited by 155
  38. 2020Learning the Best Pooling Strategy for Visual Semantic Embeddingcited by 136

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 3; 2024: 3; 2025: 2.

12 papers.

  1. 2022Visual Prompt Tuningcited by 1,674
  2. 2022OmniVL:One Foundation Model for Image-Language and Video-Language Taskscited by 143
  3. 2023Towards Efficient Visual Adaption via Structural Re-parameterizationcited by 73
  4. 2023E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuningcited by 70
  5. 2024Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning?cited by 31
  6. 2024Visual Fourier Prompt Tuningcited by 30
  7. 2023SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channelscited by 20
  8. 2024M2PT: Multimodal Prompt Tuning for Zero-shot Instruction Learningcited by 16
  9. 2021Rethinking Nearest Neighbors for Visual Classificationcited by 6
  10. 2025MEPT: Mixture of Expert Prompt Tuning as a Manifold Mappercited by 5
  11. 2022Searching for Structure in Unfalsifiable Claimscited by 4
  12. 2025Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understandingcited by 3