mlsanityData to 14 Oct 2026

Learning Visual N-Grams from Web Data

arXiv 1612.09161 ↗First posted 29 Dec 2016cs.CVcited by 95

Not on the map. Why?

Abstract (arXiv)

Real-world image recognition systems need to recognize tens of thousands of classes that constitute a plethora of visual concepts. The traditional approach of annotating thousands of images per class for training is infeasible in such a scenario, prompting the use of webly supervised data. This paper explores the training of image-recognition systems on large numbers of images and associated user comments. In particular, we develop visual n-gram models that can predict arbitrary phrases that are relevant to the content of an image. Our visual n-gram models are feed-forward convolutional networks trained using new loss functions that are inspired by n-gram models commonly used in language modeling. We demonstrate the merits of our models in phrase prediction, phrase-based image retrieval, relating images and captions, and zero-shot transfer.

Key facts. Cited by 95 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 21; 2022: 28; 2023: 28; 2024: 9; 2025: 8; 2026: 1.

95 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  4. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  5. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  6. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
  7. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
  8. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 436
  9. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
  10. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  11. 2022Prompt Distribution Learningcited by 235
  12. 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Modelcited by 223
38 more
  1. 2023Improving CLIP Training with Language Rewritescited by 217
  2. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  3. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 180
  4. 2021Align and Prompt: Video-and-Language Pre-training with Entity Promptscited by 149
  5. 2022ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Modelscited by 141
  6. 2023Hyperbolic Image-Text Representationscited by 136
  7. 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensingcited by 125
  8. 2021Multimodal Contrastive Training for Visual Representation Learningcited by 124
  9. 2022Task Residual for Tuning Vision-Language Modelscited by 123
  10. 2022Revisiting Weakly Supervised Pre-Training of Visual Perception Modelscited by 116
  11. 2022DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotationscited by 102
  12. 2022The CLEAR Benchmark: Continual LEArning on Real-World Imagerycited by 93
  13. 2023Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Modelscited by 91
  14. 2023Image Captioners Are Scalable Vision Learners Toocited by 89
  15. 2022Debiased Learning from Naturally Imbalanced Pseudo-Labelscited by 85
  16. 2022K-LITE: Learning Transferable Visual Models with External Knowledgecited by 79
  17. 2023An Inverse Scaling Law for CLIP Trainingcited by 77
  18. 2021Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Searchcited by 57
  19. 2022Multitask Vision-Language Prompt Tuningcited by 51
  20. 2023Enhancing CLIP with GPT-4: Harnessing Visual Descriptions as Promptscited by 50
  21. 2022Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Biascited by 43
  22. 2022American == White in Multimodal Language-and-Image AIcited by 39
  23. 2021Supervised Transfer Learning at Scale for Medical Imagingcited by 38
  24. 2021Data Efficient Language-supervised Zero-shot Recognition with Optimal Transport Distillationcited by 36
  25. 2021Assessing Multilingual Fairness in Pre-trained Multimodal Representationscited by 35
  26. 2024LocCa: Visual Pretraining with Location-aware Captionerscited by 35
  27. 2024Disentangled 3D Scene Generation with Layout Learningcited by 33
  28. 2023Learning Visual Representations via Language-Guided Samplingcited by 32
  29. 2023Correlation-Guided Query-Dependency Calibration for Video Temporal Groundingcited by 31
  30. 2022Evidence for Hypodescent in Visual Semantic AIcited by 30
  31. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
  32. 2022No Token Left Behind: Explainability-Aided Image Classification and Generationcited by 25
  33. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  34. 2022Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Modelscited by 22
  35. 2023DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotationscited by 22
  36. 2022Knowledge Distillation as Efficient Pre-training: Faster Convergence, Higher Data-efficiency, and Better Transferabilitycited by 21
  37. 2023From Text to Pixels: A Context-Aware Semantic Synergy Solution for Infrared and Visible Image Fusioncited by 20
  38. 2024SignCLIP: Connecting Text and Sign Language by Contrastive Learningcited by 20

50 most cited of 95 All 95 (JSON)