# Towards Context-aware Interaction Recognition

arXiv: https://arxiv.org/abs/1703.06246 · HTML: https://mlsanity.com/paper/1703.06246/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2017.

## Built on by (most cited first, 20 of 20)

- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [ReVersion: Diffusion-Based Relation Inversion from Images](https://mlsanity.com/paper/2303.13495/) (2023)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation](https://mlsanity.com/paper/2103.05271/) (2021)
- [Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queries](https://mlsanity.com/paper/2105.02170/) (2021)
- [Consensus Graph Representation Learning for Better Grounded Image Captioning](https://mlsanity.com/paper/2112.00974/) (2021)
- [HL-Net: Heterophily Learning Network for Scene Graph Generation](https://mlsanity.com/paper/2205.01316/) (2022)
- [Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation](https://mlsanity.com/paper/2111.13517/) (2021)
- [Relational Graph Learning for Grounded Video Description Generation](https://mlsanity.com/paper/2112.00967/) (2021)
- [ACP++: Action Co-occurrence Priors for Human-Object Interaction Detection](https://mlsanity.com/paper/2109.04047/) (2021)
- [Object-Centric Representation Learning for Video Question Answering](https://mlsanity.com/paper/2104.05166/) (2021)
- [2.5D Visual Relationship Detection](https://mlsanity.com/paper/2104.12727/) (2021)
- [Disentangled Action Recognition with Knowledge Bases](https://mlsanity.com/paper/2207.01708/) (2022)
- [Neural Message Passing for Visual Relationship Detection](https://mlsanity.com/paper/2208.04165/) (2022)
- [Distance-Aware Occlusion Detection with Focused Attention](https://mlsanity.com/paper/2208.11122/) (2022)
- [What and When to Look?: Temporal Span Proposal Network for Video Relation Detection](https://mlsanity.com/paper/2107.07154/) (2021)
- [ViRED: Prediction of Visual Relations in Engineering Drawings](https://mlsanity.com/paper/2409.00909/) (2024)
- [SCoRD: Subject-Conditional Relation Detection with Text-Augmented Data](https://mlsanity.com/paper/2308.12910/) (2023)
- [RelVAE: Generative Pretraining for few-shot Visual Relationship Detection](https://mlsanity.com/paper/2311.16261/) (2023)
- [Symbolic Graph Inference for Compound Scene Understanding](https://mlsanity.com/paper/2410.22626/) (2024)

## Abstract (arXiv)

Recognizing how objects interact with each other is a crucial task in visual recognition. If we define the context of the interaction to be the objects involved, then most current methods can be categorized as either: (i) training a single classifier on the combination of the interaction and its context; or (ii) aiming to recognize the interaction independently of its explicit context. Both methods suffer limitations: the former scales poorly with the number of combinations and fails to generalize to unseen combinations, while the latter often leads to poor interaction recognition performance due to the difficulty of designing a context-independent interaction classifier. To mitigate those drawbacks, this paper proposes an alternative, context-aware interaction recognition framework. The key to our method is to explicitly construct an interaction classifier which combines the context, and the interaction. The context is encoded via word2vec into a semantic space, and is used to derive a classification result for the interaction. The proposed method still builds one classifier for one interaction (as per type (ii) above), but the classifier built is adaptive to context via weights which are context dependent. The benefit of using the semantic space is that it naturally leads to zero-shot generalizations in which semantically similar contexts (subjectobject pairs) can be recognized as suitable contexts for an interaction, even if they were not observed in the training set.