mlsanityData to 14 Oct 2026

PRISM: Predictive Recomposition via Semantic Latent Decomposition for View-invariant Video Representation Learning

arXiv 2608.30388 ↗First posted 31 Aug 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Cross-view video representation learning aims to capture viewpoint-invariant action semantics despite substantial appearance changes across egocentric and exocentric videos. However, existing methods encode each video as a unified embedding, where view-invariant and view-variant semantics inevitably entangle under co-occurrences - a failure mode we show persists even in cross-view methods explicitly trained for view-invariance. Our key insight is that a view-invariant feature is truly disentangled when it can be sufficiently recomposed with an arbitrary view-variant feature while preserving their independent semantics. Building on this, we propose PRISM, that decomposes video into view-invariant and view-variant latents and recompose them under language supervision encouraging clean decomposition of the two streams. PRISM achieves state-of-the-art results on EgoExo4D, EgoExoLearn, AE2, even surpassing in-domain models under zero-shot setting. Code is available at https://github.com/litcoderr/prism.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 27 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

17 of 27 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  4. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  5. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  6. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  7. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  8. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  9. 2019Temporal Cycle-Consistency Learningcited by 154
  10. 2018Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videoscited by 128
  11. 2018Actor and Observer: Joint Modeling of First and Third-Person Videoscited by 103
  12. 2023Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignmentcited by 46
5 more
  1. 2021Representation Learning via Global Temporal Alignment and Cycle-Consistencycited by 44
  2. 2022Frame-wise Action Representations for Long Videos via Sequence Contrastive Learningcited by 27
  3. 2025MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representationscited by 22
  4. 2023Learning from Semantic Alignment between Unpaired Multiviews for Egocentric Video Recognitioncited by 20
  5. 2023DEVIAS: Learning Disentangled Video Representations of Action and Scenecited by 7

Built on by

Not cited yet.