mlsanityData to 7 Oct 2026

EVA-02: A Visual Representation for Neon Genesis

arXiv 2303.11331 ↗First posted 20 Mar 2023cs.CV cs.CLcited by 377

Not on the map. Why?

Cited by 59 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We launch EVA-02, a next-generation Transformer-based visual representation pre-trained to reconstruct strong and robust language-aligned vision features via masked image modeling. With an updated plain Transformer architecture as well as extensive pre-training from an open & accessible giant CLIP vision encoder, EVA-02 demonstrates superior performance compared to prior state-of-the-art approaches across various representative vision tasks, while utilizing significantly fewer parameters and compute budgets. Notably, using exclusively publicly accessible training data, EVA-02 with only 304M parameters achieves a phenomenal 90.0 fine-tuning top-1 accuracy on ImageNet-1K val set. Additionally, our EVA-02-CLIP can reach up to 80.4 zero-shot top-1 on ImageNet-1K, outperforming the previous largest & best open-sourced CLIP with only ~1/6 parameters and ~1/6 image-text training data. We offer four EVA-02 variants in various model sizes, ranging from 6M to 304M parameters, all with impressive performance. To facilitate open access and open research, we release the complete suite of EVA-02 to the community at https://github.com/baaivision/EVA/tree/master/EVA-02.

Key facts. Cited by 377 papers in the map (data through 7 Oct 2026). Builds on 112 of its 147 references that are in the map. First posted on arXiv 20 Mar 2023.

Builds on

112 of 147 references.

References · 112

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  6. 2020Language Models are Few-Shot Learnerscited by 32,199
  7. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  8. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  9. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  10. 2022Training language models to follow instructions with human feedbackcited by 15,712
  11. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  12. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
38 more
  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2017Mask R-CNNcited by 7,259
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2016Layer Normalizationcited by 5,579
  9. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  12. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  13. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  14. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  15. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  16. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  17. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  18. 2022A ConvNet for the 2020scited by 3,545
  19. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  20. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  21. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  22. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  23. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  24. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  25. 2013Describing Textures in the Wildcited by 2,110
  26. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  27. 2020GLU Variants Improve Transformercited by 1,724
  28. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  29. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  30. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  31. 2019Natural Adversarial Examplescited by 1,401
  32. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  33. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  34. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  35. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  36. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  37. 2017Random Erasing Data Augmentationcited by 1,235
  38. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215

50 most cited of 112 All 112 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 45; 2024: 106; 2025: 132; 2026: 94.

377 papers.

Sort:

By area:

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  3. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  4. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  5. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 447
  6. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  7. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  8. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 300
  9. 2025YOLOv12: Attention-Centric Real-Time Object Detectorscited by 300
  10. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
  11. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  12. 2024Sapiens: Foundation for Human Vision Modelscited by 229
38 more
  1. 2024Rotary Position Embedding for Vision Transformercited by 226
  2. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
  3. 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 167
  4. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 142
  5. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoderscited by 137
  6. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 130
  7. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 125
  8. 2024Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detectioncited by 122
  9. 20234M: Massively Multimodal Masked Modelingcited by 118
  10. 2023The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open Worldcited by 114
  11. 2023SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understandingcited by 111
  12. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  13. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 103
  14. 2024EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameterscited by 99
  15. 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 85
  16. 2025Describe Anything: Detailed Localized Image and Video Captioningcited by 84
  17. 2023Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentationcited by 82
  18. 2024MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizercited by 80
  19. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  20. 2023Learning Vision from Models Rivals Learning Vision from Datacited by 71
  21. 2024Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performancecited by 70
  22. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 69
  23. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 66
  24. 2025Generalized Trajectory Scoring for End-to-end Multimodal Planningcited by 63
  25. 2025Your ViT is Secretly an Image Segmentation Modelcited by 61
  26. 2023ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulationcited by 59
  27. 2023Aligning and Prompting Everything All at Once for Universal Visual Perceptioncited by 59
  28. 2023ViTMatte: Boosting Image Matting with Pretrained Plain Vision Transformerscited by 56
  29. 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generationcited by 56
  30. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  31. 2024Reconstructive Visual Instruction Tuningcited by 54
  32. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 54
  33. 2025TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generationcited by 50
  34. 2023CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detectioncited by 46
  35. 2024Foundation Models for Video Understanding: A Surveycited by 46
  36. 2024Jina CLIP: Your CLIP Model Is Also Your Text Retrievercited by 46
  37. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  38. 2023StyleGAN knows Normal, Depth, Albedo, and Morecited by 44

50 most cited of 377 All 377 (JSON)