# Compositional Zero-Shot Learning for Attribute-Based Object Reference in Human-Robot Interaction

arXiv: https://arxiv.org/abs/2312.13655 · HTML: https://mlsanity.com/paper/2312.13655/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 21 Dec 2023.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2101.12609/) (2021)
- [Symmetry and Group in Attribute-Object Compositions](https://mlsanity.com/paper/2004.00587/) (2020)
- [Disentangling Visual Embeddings for Attributes and Objects](https://mlsanity.com/paper/2205.08536/) (2022)
- [Learning Graph Embeddings for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2105.01017/) (2021)
- [Learning Attention as Disentangler for Compositional Zero-shot Learning](https://mlsanity.com/paper/2303.15111/) (2023)
- [Learning Invariant Visual Representations for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2206.00415/) (2022)
- [Relation-aware Compositional Zero-shot Learning for Attribute-Object Pair Recognition](https://mlsanity.com/paper/2108.04603/) (2021)

## Abstract (arXiv)

Language-enabled robots have been widely studied over the past years to enable natural human-robot interaction and teaming in various real-world applications. Language-enabled robots must be able to comprehend referring expressions to identify a particular object from visual perception using a set of referring attributes extracted from natural language. However, visual observations of an object may not be available when it is referred to, and the number of objects and attributes may also be unbounded in open worlds. To address the challenges, we implement an attribute-based compositional zero-shot learning method that uses a list of attributes to perform referring expression comprehension in open worlds. We evaluate the approach on two datasets including the MIT-States and the Clothing 16K. The preliminary experimental results show that our implemented approach allows a robot to correctly identify the objects referred to by human commands.