mlsanityData to 14 Oct 2026

B-RIGHT: Benchmark Re-evaluation for Integrity in Generalized Human-Object Interaction Testing

arXiv 2501.16724 ↗First posted 28 Jan 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Human-object interaction (HOI) is an essential problem in artificial intelligence (AI) which aims to understand the visual world that involves complex relationships between humans and objects. However, current benchmarks such as HICO-DET face the following limitations: (1) severe class imbalance and (2) varying number of train and test sets for certain classes. These issues can potentially lead to either inflation or deflation of model performance during evaluation, ultimately undermining the reliability of evaluation scores. In this paper, we propose a systematic approach to develop a new class-balanced dataset, Benchmark Re-evaluation for Integrity in Generalized Human-object Interaction Testing (B-RIGHT), that addresses these imbalanced problems. B-RIGHT achieves class balance by leveraging balancing algorithm and automated generation-and-filtering processes, ensuring an equal number of instances for each HOI class. Furthermore, we design a balanced zero-shot test set to systematically evaluate models on unseen scenario. Re-evaluating existing models using B-RIGHT reveals substantial the reduction of score variance and changes in performance rankings compared to conventional HICO-DET. Our experiments demonstrate that evaluation under balanced conditions ensure more reliable and fair model comparisons.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 47 of its 56 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

47 of 56 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  10. 2020Denoising Diffusion Implicit Modelscited by 9,781
  11. 2023Visual Instruction Tuningcited by 7,869
  12. 2020End-to-End Object Detection with Transformerscited by 7,414
35 more
  1. 2013Auto-Encoding Variational Bayescited by 4,053
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  6. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 573
  7. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  8. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
  9. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  10. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 310
  11. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 266
  12. 2017Learning to Detect Human-Object Interactionscited by 240
  13. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 197
  14. 2015Visual Semantic Role Labelingcited by 190
  15. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  16. 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 120
  17. 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
  18. 2021Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformercited by 91
  19. 2023Knowing Where to Focus: Event-aware Transformer for Video Groundingcited by 79
  20. 2023HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Modelscited by 61
  21. 2023Exploring Predicate Visual Context in Detecting Human-Object Interactionscited by 56
  22. 2023RLIPv2: Fast Scaling of Relational Language-Image Pre-trainingcited by 45
  23. 2024SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detectioncited by 44
  24. 2020HAA500: Human-Centric Atomic Action Dataset with Curated Videoscited by 39
  25. 2023Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Modelscited by 34
  26. 2023Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Datacited by 32
  27. 2024Open-World Human-Object Interaction Detection via Multi-modal Promptscited by 28
  28. 2024VLM-PL: Advanced Pseudo Labeling Approach for Class Incremental Object Detection via Vision-Language Modelcited by 23
  29. 2023Boosting Human-Object Interaction Detection with Text-to-Image Diffusion Modelcited by 22
  30. 2023Category Query Learning for Human-Object Interaction Classificationcited by 21
  31. 2023Diagnosing Human-object Interaction Detectorscited by 11
  32. 2024Disentangled Pre-training for Human-Object Interaction Detectioncited by 10
  33. 2022Chairs Can be Stood on: Overcoming Object Bias in Human-Object Interaction Detectioncited by 9
  34. 2023Improving Human-Object Interaction Detection via Virtual Image Learningcited by 4
  35. 2020Diagnosing Rarity in Human-Object Interaction Detectioncited by 3

Built on by

Not cited yet.