mlsanityData to 7 Oct 2026

An analysis of HOI: using a training-free method with multimodal visual foundation models when only the test set is available, without the training set

arXiv 2408.05772 ↗First posted 11 Aug 2024cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Human-Object Interaction (HOI) aims to identify the pairs of humans and objects in images and to recognize their relationships, ultimately forming langle human, object, verb rangle triplets. Under default settings, HOI performance is nearly saturated, with many studies focusing on long-tail distribution and zero-shot/few-shot scenarios. Let us consider an intriguing problem:``What if there is only test dataset without training dataset, using multimodal visual foundation model in a training-free manner? '' This study uses two experimental settings: grounding truth and random arbitrary combinations. We get some interesting conclusion and find that the open vocabulary capabilities of the multimodal visual foundation model are not yet fully realized. Additionally, replacing the feature extraction with grounding DINO further confirms these findings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 22 references that are in the map. First posted on arXiv 11 Aug 2024.

Builds on

17 of 22 references.

References · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  4. 2017Learning to Detect Human-Object Interactionscited by 238
  5. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  6. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  7. 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 119
  8. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  9. 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
  10. 2020Detailed 2D-3D Joint Representation for Human-Object Interactioncited by 93
  11. 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
  12. 2020Detecting Human-Object Interactions with Action Co-occurrence Priorscited by 72
5 more
  1. 2022RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detectioncited by 54
  2. 2022Interactiveness Field in Human-Object Interactionscited by 40
  3. 2022Mining Cross-Person Cues for Body-Part Interactiveness Learning in HOI Detectioncited by 34
  4. 2022End-to-End Zero-Shot HOI Detection via Vision and Language Knowledge Distillationcited by 30
  5. 2023Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learningcited by 11

Built on by

Not cited yet.