mlsanityData to 14 Oct 2026

Human-centric Spatio-Temporal Video Grounding With Visual Transformers

arXiv 2011.05049 ↗First posted 10 Nov 2020cs.CV cs.AI cs.MMcited by 85

Not on the map. Why?

Cited by 22 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this work, we introduce a novel task - Humancentric Spatio-Temporal Video Grounding (HC-STVG). Unlike the existing referring expression tasks in images or videos, by focusing on humans, HC-STVG aims to localize a spatiotemporal tube of the target person from an untrimmed video based on a given textural description. This task is useful, especially for healthcare and security-related applications, where the surveillance videos can be extremely long but only a specific person during a specific period of time is concerned. HC-STVG is a video grounding task that requires both spatial (where) and temporal (when) localization. Unfortunately, the existing grounding methods cannot handle this task well. We tackle this task by proposing an effective baseline method named Spatio-Temporal Grounding with Visual Transformers (STGVT), which utilizes Visual Transformers to extract cross-modal representations for video-sentence matching and temporal localization. To facilitate this task, we also contribute an HC-STVG dataset consisting of 5,660 video-sentence pairs on complex multi-person scenes. Specifically, each video lasts for 20 seconds, pairing with a natural query sentence with an average of 17.25 words. Extensive experiments are conducted on this dataset, demonstrating the newly-proposed method outperforms the existing baseline methods.

Key facts. Cited by 85 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 5; 2023: 9; 2024: 14; 2025: 26; 2026: 23.

85 papers.

Sort:

By area:

  1. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
  2. 2023Video Understanding with Large Language Models: A Surveycited by 206
  3. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  4. 2021A Survey of Visual Transformerscited by 117
  5. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 108
  6. 2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 98
  7. 2021Negative Sample Matters: A Renaissance of Metric Learning for Temporal Groundingcited by 90
  8. 2025Describe Anything: Detailed Localized Image and Video Captioningcited by 84
  9. 2024VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLMcited by 72
  10. 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 68
  11. 2024Toward Visual Grounding: A Surveycited by 52
  12. 2023PG-Video-LLaVA: Pixel Grounding Large Video-Language Modelscited by 51
38 more
  1. 2024VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videoscited by 51
  2. 2025Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videoscited by 51
  3. 2024Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editingcited by 49
  4. 2023Panoptic Video Scene Graph Generationcited by 46
  5. 2024Context-Guided Spatio-Temporal Video Groundingcited by 46
  6. 2022Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Groundingcited by 42
  7. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  8. 2025UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spacescited by 36
  9. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 35
  10. 2023Video-GroundingDINO: Towards Open-Vocabulary Spatio-Temporal Video Groundingcited by 32
  11. 2021A Survey on Temporal Sentence Grounding in Videoscited by 29
  12. 2025SpaceVLLM: Endowing Multimodal Large Language Model with Spatio-Temporal Video Grounding Capabilitycited by 28
  13. 2024Artemis: Towards Referential Understanding in Complex Videoscited by 25
  14. 2025PixelRefer: A Unified Framework for Spatio-Temporal Object Referring with Arbitrary Granularitycited by 21
  15. 2022End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Groundingcited by 19
  16. 2025Knowing Your Target: Target-Aware Transformer Makes Better Spatio-Temporal Video Groundingcited by 18
  17. 2025Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Markscited by 17
  18. 2025NVIDIA Nemotron Nano V2 VLcited by 17
  19. 2021Augmented 2D-TAN: A Two-stage Approach for Human-centric Spatio-Temporal Video Groundingcited by 13
  20. 20244-LEGS: 4D Language Embedded Gaussian Splattingcited by 12
  21. 2024VideoOrion: Tokenizing Object Dynamics in Videoscited by 12
  22. 2025STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Groundingcited by 11
  23. 2025Fine-grained Spatiotemporal Grounding on Egocentric Videoscited by 11
  24. 2025RynnEC: Bringing MLLMs into Embodied Worldcited by 11
  25. 2023What, when, and where? -- Self-Supervised Spatio-Temporal Grounding in Untrimmed Multi-Action Videos from Narrated Instructionscited by 10
  26. 2023Spatial-Temporal Knowledge-Embedded Transformer for Video Scene Graph Generationcited by 10
  27. 2025Contextual Self-paced Learning for Weakly Supervised Spatio-Temporal Video Groundingcited by 10
  28. 2025Unleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Groundingcited by 9
  29. 20212rd Place Solutions in the HC-STVG track of Person in Context Challenge 2021cited by 8
  30. 2025ToG-Bench: Task-Oriented Spatio-Temporal Grounding in Egocentric Videoscited by 8
  31. 2025OmniSTVG: Toward Spatio-Temporal Omni-Object Video Groundingcited by 7
  32. 2026VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verificationcited by 7
  33. 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generationcited by 6
  34. 2023DETR with Additional Global Aggregation for Cross-domain Weakly Supervised Object Detectioncited by 5
  35. 2024Open-Vocabulary Spatio-Temporal Action Detectioncited by 5
  36. 2025LLaFEA: Frame-Event Complementary Fusion for Fine-Grained Spatiotemporal Understanding in LMMscited by 5
  37. 2025Large-scale Pre-training for Grounded Video Caption Generationcited by 5
  38. 2025OmniGround: A Comprehensive Spatio-Temporal Grounding Benchmark for Real-World Complex Scenarioscited by 5

50 most cited of 85 All 85 (JSON)