mlsanityData to 7 Oct 2026

Video Relation Detection with Trajectory-aware Multi-modal Features

arXiv 2101.08165 ↗First posted 20 Jan 2021cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Video relation detection problem refers to the detection of the relationship between different objects in videos, such as spatial relationship and action relationship. In this paper, we present video relation detection with trajectory-aware multi-modal features to solve this task. Considering the complexity of doing visual relation detection in videos, we decompose this task into three sub-tasks: object detection, trajectory proposal and relation prediction. We use the state-of-the-art object detection method to ensure the accuracy of object trajectory detection and multi-modal feature representation to help the prediction of relation between objects. Our method won the first place on the video relation detection task of Video Relation Understanding Grand Challenge in ACM Multimedia 2020 with 11.74% mAP, which surpasses other methods by a large margin.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 10 of its 19 references that are in the map. First posted on arXiv 20 Jan 2021.

Builds on

10 of 19 references.

References · 10

  1. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  2. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  3. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  4. 2017Focal Loss for Dense Object Detectioncited by 749
  5. 2020ResNeSt: Split-Attention Networkscited by 445
  6. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 413
  7. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
  8. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
  9. 2016Seq-NMS for Video Object Detectioncited by 68
  10. 2019Video Relationship Reasoning using Gated Spatio-Temporal Energy Graphcited by 41

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 1; 2023: 1.

8 papers.

  1. 2021Classification-Then-Grounding: Reformulating Video Scene Graphs as Temporal Bipartite Graphscited by 23
  2. 2021Social Fabric: Tubelet Compositions for Video Relation Detectioncited by 18
  3. 2021Video Relation Detection via Tracklet based Visual Transformercited by 17
  4. 2021Diagnosing Errors in Video Relation Detectorscited by 4
  5. 2021Decoupled Spatial Temporal Graphs for Generic Visual Groundingcited by 3
  6. 2023Multi-Label Meta Weighting for Long-Tailed Dynamic Scene Graph Generationcited by 3
  7. 2021What and When to Look?: Temporal Span Proposal Network for Video Relation Detectioncited by 1
  8. 2022Rethinking Multi-Modal Alignment in Video Question Answering from Feature and Sample Perspectivescited by 0