mlsanityData to 14 Oct 2026

Learning Video Representations using Contrastive Bidirectional Transformer

arXiv 1906.05743 ↗First posted 13 Jun 2019cs.LG cs.CV stat.MLcited by 39

Not on the map. Why?

Abstract (arXiv)

This paper proposes a self-supervised learning approach for video features that results in significantly improved performance on downstream tasks (such as video classification, captioning and segmentation) compared to existing methods. Our method extends the BERT model for text sequences to the case of sequences of real-valued feature vectors, by replacing the softmax loss with noise contrastive estimation (NCE). We also show how to learn representations from sequences of visual features and sequences of words derived from ASR (automatic speech recognition), and show that such cross-modal training (when possible) helps even more.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 25; 2022: 8; 2023: 2; 2024: 2; 2025: 1; 2026: 1.

39 papers.

Sort:

By area:

  1. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  2. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  3. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  4. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  5. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  6. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  7. 2021Anticipative Video Transformercited by 200
  8. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  9. 2022Pretrained Language Models for Text Generation: A Surveycited by 122
  10. 2021VLM: Task-agnostic Video-Language Model Pre-training for Video Understandingcited by 102
  11. 2021Pretrained Language Models for Text Generation: A Surveycited by 95
  12. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
27 more
  1. 2021Robust Audio-Visual Instance Discriminationcited by 80
  2. 2022Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-trainingcited by 72
  3. 2022Video Transformers: A Surveycited by 63
  4. 2021Learning the Predictability of the Futurecited by 49
  5. 2021Playing Lottery Tickets with Vision and Languagecited by 45
  6. 2021Representation Learning via Global Temporal Alignment and Cycle-Consistencycited by 44
  7. 2021Time-Equivariant Contrastive Video Representation Learningcited by 44
  8. 2021Space-Time Crop & Attend: Improving Cross-modal Video Representation Learningcited by 31
  9. 2022Learning to Answer Visual Questions from Web Videoscited by 30
  10. 2021Self-supervised object detection from audio-visual correspondencecited by 29
  11. 2021Which Mutual-Information Representation Learning Objectives are Sufficient for Control?cited by 29
  12. 2023Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipationcited by 21
  13. 2021Unsupervised Visual Representation Learning by Tracking Patches in Videocited by 20
  14. 2021Look at What I'm Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videoscited by 18
  15. 2021GEM: A General Evaluation Benchmark for Multimodal Taskscited by 16
  16. 2022RepsNet: Combining Vision with Language for Automated Medical Reportscited by 14
  17. 2021Composable Augmentation Encoding for Video Representation Learningcited by 13
  18. 2024Unlocking Exocentric Video-Language Data for Egocentric Video Representation Learningcited by 12
  19. 2021MaCLR: Motion-aware Contrastive Learning of Representations for Videoscited by 10
  20. 2021Contrastive Learning of Global-Local Video Representationscited by 6
  21. 2024Learning text-to-video retrieval from image captioningcited by 6
  22. 2022End-to-End Multimodal Representation Learning for Video Dialogcited by 5
  23. 2022DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attentioncited by 4
  24. 2021InfoBehavior: Self-supervised Representation Learning for Ultra-long Behavior Sequence via Hierarchical Groupingcited by 1
  25. 2023Nearest-Neighbor Inter-Intra Contrastive Learning from Unlabeled Videoscited by 0
  26. 2025Optimizing Sensory Neurons: Nonlinear Attention Mechanisms for Accelerated Convergence in Permutation-Invariant Neural Networks for Reinforcement Learningcited by 0
  27. 2026CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Modelscited by 0