mlsanityData to 7 Oct 2026

Auto-captions on GIF: A Large-scale Video-sentence Dataset for Vision-language Pre-training

arXiv 2007.02375 ↗First posted 5 Jul 2020cs.CV cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

In this work, we present Auto-captions on GIF, which is a new large-scale pre-training dataset for generic video understanding. All video-sentence pairs are created by automatically extracting and filtering video caption annotations from billions of web pages. Auto-captions on GIF dataset can be utilized to pre-train the generic feature representation or encoder-decoder structure for video captioning, and other downstream tasks (e.g., sentence localization in videos, video question answering, etc.) as well. We present a detailed analysis of Auto-captions on GIF dataset in comparison to existing video-sentence datasets. We also provide an evaluation of a Transformer-based encoder-decoder structure for vision-language pre-training, which is further adapted to video captioning downstream task and yields the compelling generalizability on MSR-VTT. The dataset is available at urlhttp://www.auto-video-captions.top/2020/dataset.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 10; 2022: 7; 2023: 3; 2024: 1; 2025: 1.

22 papers.

Sort:

By area:

  1. 2021Trustworthy AI: From Principles to Practicescited by 206
  2. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  3. 2021CLIP4Caption: CLIP for Video Captioncited by 88
  4. 2022Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detectioncited by 46
  5. 2022Semantic-Conditional Diffusion Networks for Image Captioningcited by 38
  6. 2022Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrievalcited by 34
  7. 2021A Survey on Temporal Sentence Grounding in Videoscited by 29
  8. 2021Survey: Transformer based Video-Language Pre-trainingcited by 27
  9. 2021Understanding Chinese Video and Language via Contrastive Multimodal Pre-Trainingcited by 23
  10. 2021Scheduled Sampling in Vision-Language Pretraining with Decoupled Encoder-Decoder Networkcited by 17
  11. 2021CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoisingcited by 17
  12. 20233DStyle-Diffusion: Pursuing Fine-grained Text-driven 3D Stylization with 2D Diffusion Modelscited by 17
10 more
  1. 2021X-modaler: A Versatile and High-performance Codebase for Cross-modal Analyticscited by 10
  2. 2022Uni-EDEN: Universal Encoder-Decoder Network by Multi-Granular Vision-Language Pre-trainingcited by 7
  3. 2022Bi-Calibration Networks for Weakly-Supervised Video Representation Learningcited by 4
  4. 2023View while Moving: Efficient Video Recognition in Long-untrimmed Videoscited by 4
  5. 2025Learning Sparsity for Effective and Efficient Music Performance Question Answeringcited by 4
  6. 2022Silver-Bullet-3D at ManiSkill 2021: Learning-from-Demonstrations and Heuristic Rule-based Methods for Object Manipulationcited by 3
  7. 2023Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Datacited by 3
  8. 2021Synchronized Audio-Visual Frames with Fractional Positional Encoding for Transformers in Video-to-Text Translationcited by 2
  9. 2021Extended Self-Critical Pipeline for Transforming Videos to Text (TRECVID-VTT Task 2021) -- Team: MMCUniAugsburgcited by 1
  10. 2024SNP-S3: Shared Network Pre-training and Significant Semantic Strengthening for Various Video-Text Taskscited by 1