mlsanityData to 14 Oct 2026

InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions

arXiv 2211.05778 ↗First posted 10 Nov 2022cs.CVcited by 482

Not on the map. Why?

Abstract (arXiv)

Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early state. This work presents a new large-scale CNN-based foundation model, termed InternImage, which can obtain the gain from increasing parameters and training data like ViTs. Different from the recent CNNs that focus on large dense kernels, InternImage takes deformable convolution as the core operator, so that our model not only has the large effective receptive field required for downstream tasks such as detection and segmentation, but also has the adaptive spatial aggregation conditioned by input and task information. As a result, the proposed InternImage reduces the strict inductive bias of traditional CNNs and makes it possible to learn stronger and more robust patterns with large-scale parameters from massive data like ViTs. The effectiveness of our model is proven on challenging benchmarks including ImageNet, COCO, and ADE20K. It is worth mentioning that InternImage-H achieved a new record 65.4 mAP on COCO test-dev and 62.9 mIoU on ADE20K, outperforming current leading CNNs and ViTs. The code will be released at https://github.com/OpenGVLab/InternImage.

Key facts. Cited by 482 papers in the map (data through 14 Oct 2026). Builds on 90 of its 107 references that are in the map. First posted on arXiv 10 Nov 2022.

Builds on

90 of 107 references.

References · 90

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  9. 2017Mask R-CNNcited by 7,277
  10. 2020Scaling Laws for Neural Language Modelscited by 6,586
  11. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  12. 2016Layer Normalizationcited by 5,669
38 more
  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  4. 2014Going Deeper with Convolutionscited by 4,241
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  6. 2022A ConvNet for the 2020scited by 3,579
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  9. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  10. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  11. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  12. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  13. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  14. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  15. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  16. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  17. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  18. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  19. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  20. 2017Deformable Convolutional Networkscited by 1,488
  21. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  22. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  23. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  24. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  25. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  26. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  27. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  28. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  29. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  30. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  31. 2021Scaling Vision Transformerscited by 999
  32. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  33. 2020On Layer Normalization in the Transformer Architecturecited by 922
  34. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  35. 2021Florence: A New Foundation Model for Computer Visioncited by 804
  36. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  37. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 751
  38. 2021Scaling Vision with Sparse Mixture of Expertscited by 747

50 most cited of 90 All 90 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 143; 2024: 153; 2025: 118; 2026: 60.

482 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  3. 2024Depth Anything V2cited by 1,422
  4. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  5. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
  6. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  7. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  8. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  9. 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
  10. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  11. 2023Recognize Anything: A Strong Image Tagging Modelcited by 309
  12. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
38 more
  1. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 274
  2. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  3. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
  4. 2023mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Videocited by 154
  5. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 150
  6. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  7. 2024MambaOut: Do We Really Need Mamba for Vision?cited by 143
  8. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  9. 2023On the Challenges and Perspectives of Foundation Models for Medical Image Analysiscited by 135
  10. 2023ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformationcited by 121
  11. 2023ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalitiescited by 118
  12. 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 116
  13. 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
  14. 2023The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open Worldcited by 116
  15. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  16. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 110
  17. 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 104
  18. 2023Contextual Object Detection with Multimodal Large Language Modelscited by 100
  19. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 99
  20. 2024Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architecturescited by 98
  21. 2023InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Languagecited by 96
  22. 2023Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modelingcited by 94
  23. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 93
  24. 2023InceptionNeXt: When Inception Meets ConvNeXtcited by 86
  25. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  26. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 73
  27. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 70
  28. 2023RMT: Retentive Networks Meet Vision Transformerscited by 68
  29. 2024ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictionscited by 66
  30. 2024MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretrainingcited by 66
  31. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  32. 2023UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognitioncited by 64
  33. 2024Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Modelcited by 62
  34. 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
  35. 2023TransNeXt: Robust Foveal Visual Perception for Vision Transformerscited by 61
  36. 2024Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Trainingcited by 59
  37. 2023PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmcited by 56
  38. 2023Towards Label-free Scene Understanding by Vision Foundation Modelscited by 54

50 most cited of 482 All 482 (JSON)