mlsanityData to 14 Oct 2026

UnLoc: A Unified Framework for Video Localization Tasks

arXiv 2308.11062 ↗First posted 21 Aug 2023cs.CV cs.LGcited by 66

Not on the map. Why?

Abstract (arXiv)

While large-scale image-text pretrained models such as CLIP have been used for multiple video-level tasks on trimmed videos, their use for temporal localization in untrimmed videos is still a relatively unexplored task. We design a new approach for this called UnLoc, which uses pretrained image and text towers, and feeds tokens to a video-text fusion model. The output of the fusion module are then used to construct a feature pyramid in which each level connects to a head to predict a per-frame relevancy score and start/end time displacements. Unlike previous works, our architecture enables Moment Retrieval, Temporal Localization, and Action Segmentation with a single stage model, without the need for action proposals, motion based pretrained features or representation masking. Unlike specialized models, we achieve state of the art results on all three different localization tasks with a unified approach. Code will be available at: urlhttps://github.com/google-research/scenic.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Builds on 75 of its 90 references that are in the map. First posted on arXiv 21 Aug 2023.

Builds on

75 of 90 references.

References · 75

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2016Layer Normalizationcited by 5,669
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  9. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  11. 2018SlowFast Networks for Video Recognitioncited by 1,867
  12. 2021ViViT: A Video Vision Transformercited by 1,620
38 more
  1. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  2. 2016Deep Networks with Stochastic Depthcited by 979
  3. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  4. 2017Dense-Captioning Events in Videoscited by 793
  5. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  6. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  7. 2017Localizing Moments in Video with Natural Languagecited by 663
  8. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
  9. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  10. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  11. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  12. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
  13. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 507
  14. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 473
  15. 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 459
  16. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 387
  17. 2017Soft-NMS -- Improving Object Detection With One Line of Codecited by 382
  18. 2019BMN: Boundary-Matching Network for Temporal Action Proposal Generationcited by 329
  19. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 321
  20. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 320
  21. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
  22. 2018Video Action Transformer Networkcited by 316
  23. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 311
  24. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 310
  25. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 310
  26. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 305
  27. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285
  28. 2018BSN: Boundary Sensitive Network for Temporal Action Proposal Generationcited by 271
  29. 2017Temporal Action Detection with Structured Segment Networkscited by 267
  30. 2020ActBERT: Learning Global-Local Video-Text Representationscited by 267
  31. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  32. 2020UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generationcited by 259
  33. 2019G-TAD: Sub-Graph Localization for Temporal Action Detectioncited by 249
  34. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
  35. 2016The THUMOS Challenge on Action Recognition for Videos "in the Wild"cited by 213
  36. 2021Decoupling Zero-Shot Semantic Segmentationcited by 199
  37. 2018Rethinking the Faster R-CNN Architecture for Temporal Action Localizationcited by 194
  38. 2020Local-Global Video-Text Interactions for Temporal Groundingcited by 188

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 28; 2025: 25; 2026: 9.

66 papers.

Sort:

By area:

  1. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 225
  2. 2024LITA: Language Instructed Temporal-Localization Assistantcited by 125
  3. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 106
  4. 2023Grounded Question-Answering in Long Egocentric Videoscited by 64
  5. 2024VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computationcited by 55
  6. 2024R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Groundingcited by 34
  7. 2023Correlation-Guided Query-Dependency Calibration for Video Temporal Groundingcited by 31
  8. 2023BAM-DETR: Boundary-Aligned Moment Detection Transformer for Temporal Sentence Grounding in Videoscited by 29
  9. 2025Universal Video Temporal Grounding with Generative Multi-modal Large Language Modelscited by 27
  10. 2024OmniVid: A Generative Framework for Universal Video Understandingcited by 22
  11. 2024Test-Time Zero-Shot Temporal Action Localizationcited by 19
  12. 2024Chrono: A Simple Blueprint for Representing Time in MLLMscited by 18
38 more
  1. 2024UniMD: Towards Unifying Moment Retrieval and Temporal Action Detectioncited by 16
  2. 2025TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMscited by 15
  3. 2023Grounding-Prompter: Prompting LLM with Multimodal Information for Temporal Sentence Grounding in Long Videoscited by 14
  4. 2025TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Actioncited by 14
  5. 2024Seq2Time: Sequential Knowledge Transfer for Video LLM Temporal Groundingcited by 12
  6. 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMscited by 12
  7. 2024Beyond Uncertainty: Evidential Deep Learning for Robust Video Temporal Groundingcited by 10
  8. 2025iMOVE: Instance-Motion-Aware Video Understandingcited by 10
  9. 2024Context-Enhanced Video Moment Retrieval with Large Language Modelscited by 9
  10. 2024Exploiting VLM Localizability and Semantics for Open Vocabulary Action Detectioncited by 8
  11. 2024ShowHowTo: Generating Scene-Conditioned Step-by-Step Visual Instructionscited by 8
  12. 2025LD-DETR: Loop Decoder DEtection TRansformer for Video Moment Retrieval and Highlight Detectioncited by 8
  13. 2025Prototypes are Balanced Units for Efficient and Effective Partially Relevant Video Retrievalcited by 8
  14. 2024Detours for Navigating Instructional Videoscited by 7
  15. 2024Localizing Events in Videos with Multimodal Queriescited by 6
  16. 2024Exploring Scalability of Self-Training for Open-Vocabulary Temporal Action Localizationcited by 6
  17. 2024Text-Enhanced Zero-Shot Action Recognition: A training-free approachcited by 6
  18. 2024Vid-Morp: Video Moment Retrieval Pretraining from Unlabeled Videos in the Wildcited by 5
  19. 2024Zero-shot Action Localization via the Confidence of Large Vision-Language Modelscited by 4
  20. 2025Weakly Supervised Video Scene Graph Generation via Natural Language Supervisioncited by 4
  21. 2025Learning Skill-Attributes for Transferable Assessment in Videocited by 4
  22. 2024SHINE: Saliency-aware HIerarchical NEgative Ranking for Compositional Temporal Groundingcited by 3
  23. 2024Described Spatial-Temporal Video Detectioncited by 3
  24. 2024VideoLights: Feature Refinement and Cross-Task Alignment Transformer for Joint Video Highlight Detection and Moment Retrievalcited by 3
  25. 2025TimeLoc: A Unified End-to-End Framework for Precise Timestamp Localization in Long Videoscited by 3
  26. 2025HierSum: A Global and Local Attention Mechanism for Video Summarizationcited by 3
  27. 2024TFCounter:Polishing Gems for Training-Free Object Countingcited by 2
  28. 2024ActPrompt: In-Domain Feature Adaptation via Action Cues for Video Temporal Groundingcited by 2
  29. 2024Language-Assisted Human Part Motion Learning for Skeleton-Based Temporal Action Segmentationcited by 2
  30. 2025Measure Twice, Cut Once: A Semantic-Oriented Approach to Video Temporal Localization with Video LLMscited by 2
  31. 2025Text-Derived Relational Graph-Enhanced Network for Skeleton-Based Action Segmentationcited by 2
  32. 2025Grounding-MD: Grounded Video-language Pre-training for Open-World Moment Detectioncited by 2
  33. 2025Zero-Shot Temporal Interaction Localization for Egocentric Videoscited by 2
  34. 2025Sparse-Dense Side-Tuner for efficient Video Temporal Groundingcited by 2
  35. 2025Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Surveycited by 2
  36. 2026CLIP-AUTT: Test-Time Personalization with Action Unit Prompting for Fine-Grained Video Emotion Recognitioncited by 2
  37. 2026LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videoscited by 2
  38. 2024Multimodal Fusion and Coherence Modeling for Video Topic Segmentationcited by 1

50 most cited of 66 All 66 (JSON)