Recognizing Unseen States of Unknown Objects by Leveraging Knowledge Graphs
Not on the map. Why?
Abstract (arXiv)
We investigate the problem of Object State Classification (OSC) as a zero-shot learning problem. Specifically, we propose the first Object-agnostic State Classification (OaSC) method that infers the state of a certain object without relying on the knowledge or the estimation of the object class. In that direction, we capitalize on Knowledge Graphs (KGs) for structuring and organizing knowledge, which, in combination with visual information, enable the inference of the states of objects in object/state pairs that have not been encountered in the method's training set. A series of experiments investigate the performance of the proposed method in various settings, against several hypotheses and in comparison with state of the art approaches for object attribute classification. The experimental results demonstrate that the knowledge of an object class is not decisive for the prediction of its state. Moreover, the proposed OaSC method outperforms existing methods in all datasets and benchmarks by a great margin.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 45 of its 75 references that are in the map. First posted on arXiv 22 Jul 2023.
Builds on
45 of 75 references.
References · 45
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2017Inductive Representation Learning on Large Graphscited by 6,320
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2019Simplifying Graph Convolutional Networkscited by 1,477
- 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
- 2021Robust fine-tuning of zero-shot modelscited by 797
- 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 538
- 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
33 more
- 2019Composition-based Multi-Relational Graph Convolutional Networkscited by 396
- 2018Graph Convolutional Networks for Text Classificationcited by 345
- 2019Abductive Commonsense Reasoningcited by 301
- 2017Feature Generating Networks for Zero-Shot Learningcited by 257
- 2014Evaluation of Output Embeddings for Fine-Grained Image Classificationcited by 166
- 2018Zero-shot Recognition via Semantic Embeddings and Knowledge Graphscited by 155
- 2018End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completioncited by 136
- 2019Task-Driven Modular Networks for Zero-Shot Compositional Learningcited by 121
- 2021Learning to Predict Visual Attributes in the Wildcited by 96
- 2018Rethinking Knowledge Graph Propagation for Zero-Shot Learningcited by 87
- 2020A causal view of compositional zero-shot recognitioncited by 78
- 2017Learning to Act Properly: Predicting and Explaining Affordances from Imagescited by 74
- 2017Graph Convolutional Encoders for Syntax-aware Neural Machine Translationcited by 70
- 2022Disentangling Visual Embeddings for Attributes and Objectscited by 55
- 2022Siamese Contrastive Embedding Network for Compositional Zero-Shot Learningcited by 55
- 2020Learning Dynamic Belief Graphs to Generalize on Text-Based Gamescited by 49
- 2021Learning Graph Embeddings for Open World Compositional Zero-Shot Learningcited by 49
- 2016Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Imagescited by 45
- 2022KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learningcited by 41
- 2022Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videoscited by 39
- 2017Attend and Interact: Higher-Order Object Interactions for Video Understandingcited by 37
- 2015Actions ~ Transformationscited by 30
- 2017Joint Discovery of Object States and Manipulation Actionscited by 28
- 2019Integrating Semantic Knowledge to Tackle Zero-shot Text Classificationcited by 28
- 2023Learning Conditional Attributes for Compositional Zero-Shot Learningcited by 28
- 2022Learning Invariant Visual Representations for Compositional Zero-Shot Learningcited by 24
- 2019Imposing Label-Relational Inductive Bias for Extremely Fine-Grained Entity Typingcited by 22
- 2021Graph Transformer Networks: Learning Meta-path Graphs to Improve GNNscited by 22
- 2020Zero-Shot Learning with Common Sense Knowledge Graphscited by 17
- 2019TGG: Transferable Graph Generation for Zero-shot and Few-shot Learningcited by 10
- 2016End-to-End Localization and Ranking for Relative Attributescited by 9
- 2022A Survey on Visual Transfer Learning using Knowledge Graphscited by 7
- 2021Detecting Object States vs Detecting Objects: A New Dataset and a Quantitative Experimental Studycited by 6
Built on by
Citing papers per year: 2024: 1.
1 paper.