mlsanityData to 7 Oct 2026

ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models

arXiv 2204.08790 ↗First posted 19 Apr 2022cs.CV cs.CL cs.LGcited by 141

Not on the map. Why?

Cited by 38 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Learning visual representations from natural language supervision has recently shown great promise in a number of pioneering works. In general, these language-augmented visual models demonstrate strong transferability to a variety of datasets and tasks. However, it remains challenging to evaluate the transferablity of these models due to the lack of easy-to-use evaluation toolkits and public benchmarks. To tackle this, we build ELEVATER (Evaluation of Language-augmented Visual Task-level Transfer), the first benchmark and toolkit for evaluating(pre-trained) language-augmented visual models. ELEVATER is composed of three components. (i) Datasets. As downstream evaluation suites, it consists of 20 image classification datasets and 35 object detection datasets, each of which is augmented with external knowledge. (ii) Toolkit. An automatic hyper-parameter tuning toolkit is developed to facilitate model evaluation on downstream tasks. (iii) Metrics. A variety of evaluation metrics are used to measure sample-efficiency (zero-shot and few-shot) and parameter-efficiency (linear probing and full model fine-tuning). ELEVATER is a platform for Computer Vision in the Wild (CVinW), and is publicly released at at https://computer-vision-in-the-wild.github.io/ELEVATER/

Key facts. Cited by 141 papers in the map (data through 7 Oct 2026). Builds on 62 of its 96 references that are in the map. First posted on arXiv 19 Apr 2022.

Builds on

62 of 96 references.

References · 62

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  12. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
38 more
  1. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  2. 2015VQA: Visual Question Answeringcited by 2,489
  3. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  4. 2013Describing Textures in the Wildcited by 2,110
  5. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  6. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  7. 2022Visual Prompt Tuningcited by 1,674
  8. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  9. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  11. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  12. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  13. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  14. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  15. 2014How transferable are features in deep neural networks?cited by 1,216
  16. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  17. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  18. 2021Grounded Language-Image Pre-trainingcited by 1,147
  19. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  20. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  21. 2020In Search of Lost Domain Generalizationcited by 1,042
  22. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  23. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  24. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  25. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  26. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  27. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  28. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 536
  29. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  30. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  31. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  32. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  33. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  34. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  35. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  36. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
  37. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  38. 2020Frustratingly Simple Few-Shot Object Detectioncited by 280

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 15; 2023: 57; 2024: 26; 2025: 28; 2026: 15.

141 papers.

Sort:

By area:

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  3. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  4. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  5. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  6. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  7. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  8. 2023Scaling Open-Vocabulary Object Detectioncited by 335
  9. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  10. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  11. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 188
  12. 2023Large Language Model Routing with Benchmark Datasetscited by 166
38 more
  1. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 158
  2. 2022FlexiViT: One Model for All Patch Sizescited by 151
  3. 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 123
  4. 2022Task Residual for Tuning Vision-Language Modelscited by 122
  5. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  6. 2024GaussCtrl: Multi-View Consistent Text-Driven 3D Gaussian Splatting Editingcited by 114
  7. 2022Chinese CLIP: Contrastive Vision-Language Pretraining in Chinesecited by 108
  8. 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
  9. 2023A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captionscited by 107
  10. 2024T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergycited by 94
  11. 2023TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritancecited by 90
  12. 2024Advancing Multimodal Medical Capabilities of Geminicited by 88
  13. 2022AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilitiescited by 84
  14. 2023V3Det: Vast Vocabulary Visual Detection Datasetcited by 81
  15. 2022K-LITE: Learning Transferable Visual Models with External Knowledgecited by 79
  16. 2023LLM-Rec: Personalized Recommendation via Prompting Large Language Modelscited by 77
  17. 2023Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Modelscited by 72
  18. 2022Teaching Structured Vision&Language Concepts to Vision&Language Modelscited by 71
  19. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  20. 2024An Open and Comprehensive Pipeline for Unified Object Grounding and Detectioncited by 64
  21. 2024The Neglected Tails in Vision-Language Modelscited by 63
  22. 2023Learning Customized Visual Models with Retrieval-Augmented Knowledgecited by 58
  23. 2025LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Modelscited by 53
  24. 2022Multitask Vision-Language Prompt Tuningcited by 51
  25. 2023Multi-modal Queried Object Detection in the Wildcited by 47
  26. 2023Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understandingcited by 46
  27. 2023Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional Understandingcited by 40
  28. 2023Described Object Detection: Liberating Object Detection with Flexible Expressionscited by 40
  29. 2023Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphscited by 39
  30. 2023Matte Anything: Interactive Natural Image Matting with Segment Anything Modelscited by 39
  31. 2023Visual Instruction Tuning with Polite Flamingocited by 38
  32. 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMscited by 38
  33. 2023Going Beyond Nouns With Vision & Language Models Using Synthetic Datacited by 33
  34. 2023Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionalitycited by 32
  35. 2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlookcited by 32
  36. 2023Enhancing CLIP with CLIP: Exploring Pseudolabeling for Limited-Label Prompt Tuningcited by 30
  37. 2023DIME-FM: DIstilling Multimodal and Efficient Foundation Modelscited by 28
  38. 2024Few-shot Adaptation of Multi-modal Foundation Models: A Surveycited by 28

50 most cited of 141 All 141 (JSON)