mlsanityData to 7 Oct 2026

InternVideo: General Video Foundation Models via Generative and Discriminative Learning

arXiv 2212.03191 ↗First posted 6 Dec 2022cs.CVcited by 441

Not on the map. Why?

Cited by 79 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The foundation models have recently shown excellent performance on a variety of downstream tasks in computer vision. However, most existing vision foundation models simply focus on image-level pretraining and adpation, which are limited for dynamic and complex video-level understanding tasks. To fill the gap, we present general video foundation models, InternVideo, by taking advantage of both generative and discriminative self-supervised video learning. Specifically, InternVideo efficiently explores masked video modeling and video-language contrastive learning as the pretraining objectives, and selectively coordinates video representations of these two complementary frameworks in a learnable manner to boost various video applications. Without bells and whistles, InternVideo achieves state-of-the-art performance on 39 video datasets from extensive tasks including video action recognition/detection, video-language alignment, and open-world video applications. Especially, our methods can obtain 91.1% and 77.2% top-1 accuracy on the challenging Kinetics-400 and Something-Something V2 benchmarks, respectively. All of these results effectively show the generality of our InternVideo for video understanding. The code will be released at https://github.com/OpenGVLab/InternVideo .

Key facts. Cited by 441 papers in the map (data through 7 Oct 2026). Builds on 102 of its 110 references that are in the map. First posted on arXiv 6 Dec 2022.

Builds on

102 of 110 references.

References · 102

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2017Mask R-CNNcited by 7,259
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
38 more
  1. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  2. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  5. 2018SlowFast Networks for Video Recognitioncited by 1,862
  6. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  7. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  8. 2021ViViT: A Video Vision Transformercited by 1,610
  9. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  10. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  11. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  12. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  13. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  14. 2016Colorful Image Colorizationcited by 1,113
  15. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  16. 2021Video Swin Transformercited by 1,090
  17. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  18. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  19. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  20. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 899
  21. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  22. 2021Robust fine-tuning of zero-shot modelscited by 785
  23. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  24. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  25. 2017Localizing Moments in Video with Natural Languagecited by 660
  26. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  27. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  28. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  29. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  30. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  31. 2015Towards Open Set Deep Networkscited by 533
  32. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  33. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 506
  34. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  35. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  36. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  37. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  38. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427

50 most cited of 102 All 102 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 94; 2024: 143; 2025: 131; 2026: 72.

441 papers.

Sort:

By area:

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  3. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  4. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  5. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  6. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  7. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  8. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  9. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  10. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  11. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  12. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
38 more
  1. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
  2. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  3. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  4. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
  5. 2023Valley: Video Assistant with Large Language model Enhanced abilitYcited by 231
  6. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 224
  7. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  8. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  9. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 200
  10. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  11. 2023OneLLM: One Framework to Align All Modalities with Languagecited by 185
  12. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 180
  13. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
  14. 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 168
  15. 2024Streaming Long Video Understanding with Large Language Modelscited by 164
  16. 2023mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Videocited by 154
  17. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  18. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 151
  19. 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 146
  20. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  21. 2023VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Datasetcited by 138
  22. 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
  23. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 133
  24. 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 126
  25. 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
  26. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  27. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 111
  28. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  29. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 106
  30. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  31. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 103
  32. 2023VDT: General-purpose Video Diffusion Transformers via Mask Modelingcited by 102
  33. 2024Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenizationcited by 98
  34. 2024An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLMcited by 97
  35. 2023InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Languagecited by 96
  36. 2023AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Modelcited by 93
  37. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  38. 2024Video ReCap: Recursive Captioning of Hour-Long Videoscited by 88

50 most cited of 441 All 441 (JSON)