mlsanityData to 14 Oct 2026

W2Rep: Learning Visual Representations by Watching the World Change

arXiv 2609.35464 ↗First posted 28 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Images capture the world at one moment, whereas video reveals how it changes. Image self-supervision learns spatial structure from a single moment, while video methods commonly learn temporal relationships inside a representation computed jointly from several frames. We ask whether watching a scene change can instead improve features available from one image without sacrificing the ability to represent video. We introduce W2Rep, a masked feature-prediction framework in which an independently encoded source image participates in prediction at the same or another moment. The predictor is conditioned on visible video context, the queried location, and the signed time interval between source and target. This gives the cross-frame objective two complementary roles: the image path learns features that remain useful across time, while the video path must gather evidence that is missing from the source image. Across model scales and downstream tasks, W2Rep improves frozen and fine-tuned recognition under our comparison protocol, while joint video encoding provides further gains over frame-wise aggregation. Controlled experiments show that these gains depend on directly updating the source-image features and on using both video context and temporal displacement. Overall, change across a video can supervise a visual encoder whose representations remain useful at either image or video granularity. Code is available at~hrefhttps://wenooi.github.io/W2Rephttps://wenooi.github.io/W2Rep.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 41 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

35 of 41 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  10. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  11. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  12. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
23 more
  1. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  2. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  3. 2024Genie: Generative Interactive Environmentscited by 747
  4. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  5. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  6. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  7. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
  8. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  9. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  10. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
  11. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  12. 2020SpeedNet: Learning the Speediness in Videoscited by 171
  13. 2021BEVT: BERT Pretraining of Video Transformerscited by 169
  14. 2019Temporal Cycle-Consistency Learningcited by 154
  15. 2016Learning Features by Watching Objects Movecited by 138
  16. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 117
  17. 2022OmniMAE: Single Model Masked Pretraining on Images and Videoscited by 99
  18. 2022Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learningcited by 90
  19. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 57
  20. 2021Long-Short Temporal Contrastive Learning of Video Transformerscited by 42
  21. 2024Visual Representation Learning with Stochastic Frame Predictioncited by 10
  22. 2026You Don't Need Strong Assumptions: Visual Representation Learning via Temporal Differencescited by 5
  23. 2025Token Bottleneck: One Token to Remember Dynamicscited by 4

Built on by

Not cited yet.