mlsanityData to 14 Oct 2026

Switch-a-View: View Selection Learned from Unlabeled In-the-wild Videos

arXiv 2412.18386 ↗First posted 24 Dec 2024cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

We introduce SWITCH-A-VIEW, a model that learns to automatically select the viewpoint to display at each timepoint when creating a how-to video. The key insight of our approach is how to train such a model from unlabeled -- but human-edited -- video samples. We pose a pretext task that pseudo-labels segments in the training videos for their primary viewpoint (egocentric or exocentric), and then discovers the patterns between the visual and spoken content in a how-to video on the one hand and its view-switch moments on the other hand. Armed with this predictor, our model can be applied to new multi-view video settings for orchestrating which viewpoint should be displayed when, even when such settings come with limited labels. We demonstrate our idea on a variety of real-world videos from HowTo100M and Ego-Exo4D, and rigorously validate its advantages. Project: https://vision.cs.utexas.edu/projects/switch_a_view/.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 39 of its 67 references that are in the map. First posted on arXiv 24 Dec 2024.

Builds on

39 of 67 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2017The Kinetics Human Action Video Datasetcited by 1,909
  7. 2018SlowFast Networks for Video Recognitioncited by 1,867
  8. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  9. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  10. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  11. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  12. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
27 more
  1. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  2. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 280
  3. 2018Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videoscited by 128
  4. 2018Actor and Observer: Joint Modeling of First and Third-Person Videoscited by 103
  5. 2021Ego-Exo: Transferring Visual Representations from Third-person to First-person Videoscited by 99
  6. 2022Learning To Recognize Procedural Activities with Distant Supervisioncited by 85
  7. 2018Video Summarization Using Fully Convolutional Sequence Networkscited by 71
  8. 2023Align and Attend: Multimodal Summarization with Dual Contrastive Lossescited by 65
  9. 2017A Dataset for Developing and Benchmarking Active Visioncited by 50
  10. 2023Procedure-Aware Pretraining for Instructional Video Understandingcited by 48
  11. 2023Learning Procedure-aware Video Representation from Instructional Videos and Their Narrationscited by 46
  12. 2022TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliencycited by 35
  13. 2023Video-Mined Task Graphs for Keystep Recognition in Instructional Videoscited by 35
  14. 2023Learning to Ground Instructional Articles in Videos through Narrationscited by 28
  15. 2016Summary Transfer: Exemplar-based Subset Selection for Video Summarizationcited by 24
  16. 2016Pano2Vid: Automatic Cinematography for Watching 360circ Videoscited by 22
  17. 2017Deep 360 Pilot: Learning a Deep Agent for Piloting through 360deg Sports Videocited by 22
  18. 2020Adaptive Video Highlight Detection by Learning from User Historycited by 22
  19. 2018Geometry-Aware Recurrent Neural Networks for Active Visual Recognitioncited by 20
  20. 2017Collaborative Summarization of Topic-Related Videoscited by 17
  21. 2019Emergence of Exploratory Look-Around Behaviors through Active Observation Completioncited by 14
  22. 2018How Local is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarizationcited by 13
  23. 2020Attend and Segment: Attention Guided Active Semantic Segmentationcited by 9
  24. 2021Glimpse-Attend-and-Explore: Self-Attention for Active Visual Explorationcited by 9
  25. 2018Multi-Stream Dynamic Video Summarizationcited by 8
  26. 2018Learning Sports Camera Selection from Internet Videoscited by 6
  27. 2017Self-view Grounding Given a Narrated 360deg Videocited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MistExit: Learning to Exit for Early Mistake Detection in Procedural Videoscited by 2