# Visual Superordinate Abstraction for Robust Concept Learning

arXiv: https://arxiv.org/abs/2205.14444 · HTML: https://mlsanity.com/paper/2205.14444/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 46 references that are in the map.
- First posted on arXiv 28 May 2022.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding](https://mlsanity.com/paper/2104.12763/) (2021)
- [From Recognition to Cognition: Visual Commonsense Reasoning](https://mlsanity.com/paper/1811.10830/) (2018)
- [The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision](https://mlsanity.com/paper/1904.12584/) (2019)
- [Unbiased Scene Graph Generation from Biased Training](https://mlsanity.com/paper/2002.11949/) (2020)
- [Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding](https://mlsanity.com/paper/1810.02338/) (2018)
- [Counterfactual VQA: A Cause-Effect Look at Language Bias](https://mlsanity.com/paper/2006.04315/) (2020)
- [Compositional Attention Networks for Machine Reasoning](https://mlsanity.com/paper/1803.03067/) (2018)
- [Inferring and Executing Programs for Visual Reasoning](https://mlsanity.com/paper/1705.03633/) (2017)
- [Learning to Reason: End-to-End Module Networks for Visual Question Answering](https://mlsanity.com/paper/1704.05526/) (2017)
- [Visual Commonsense R-CNN](https://mlsanity.com/paper/2002.12204/) (2020)
- [Two Causal Principles for Improving Visual Dialog](https://mlsanity.com/paper/1911.10496/) (2019)
- [Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning](https://mlsanity.com/paper/2103.16564/) (2021)
- [Deconfounded Image Captioning: A Causal Retrospect](https://mlsanity.com/paper/2003.03923/) (2020)
- [Explainable Neural Computation via Stack Neural Module Networks](https://mlsanity.com/paper/1807.08556/) (2018)
- [Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning](https://mlsanity.com/paper/1803.05268/) (2018)
- [Word meaning in minds and machines](https://mlsanity.com/paper/2008.01766/) (2020)
- [Visual Concept-Metaconcept Learning](https://mlsanity.com/paper/2002.01464/) (2020)
- [Causal Induction from Visual Observations for Goal Directed Tasks](https://mlsanity.com/paper/1910.01751/) (2019)
- [A Competence-aware Curriculum for Visual Concepts Learning via Question Answering](https://mlsanity.com/paper/2007.01499/) (2020)
- [Interpretable Visual Reasoning via Induced Symbolic Space](https://mlsanity.com/paper/2011.11603/) (2020)
- [Zero-Shot Fine-Grained Classification by Deep Feature Learning with Semantics](https://mlsanity.com/paper/1707.00785/) (2017)
- [On transfer learning using a MAC model variant](https://mlsanity.com/paper/1811.06529/) (2018)

## Built on by (most cited first, 1 of 1)

- [Human Inspired Progressive Alignment and Comparative Learning for Grounded Word Acquisition](https://mlsanity.com/paper/2307.02615/) (2023)

## Abstract (arXiv)

Concept learning constructs visual representations that are connected to linguistic semantics, which is fundamental to vision-language tasks. Although promising progress has been made, existing concept learners are still vulnerable to attribute perturbations and out-of-distribution compositions during inference. We ascribe the bottleneck to a failure of exploring the intrinsic semantic hierarchy of visual concepts, e.g. red, blue,... ∈ `color' subspace yet cube ∈ `shape'. In this paper, we propose a visual superordinate abstraction framework for explicitly modeling semantic-aware visual subspaces (i.e. visual superordinates). With only natural visual question answering data, our model first acquires the semantic hierarchy from a linguistic view, and then explores mutually exclusive visual superordinates under the guidance of linguistic hierarchy. In addition, a quasi-center visual concept clustering and a superordinate shortcut learning schemes are proposed to enhance the discrimination and independence of concepts within each visual superordinate. Experiments demonstrate the superiority of the proposed framework under diverse settings, which increases the overall answering accuracy relatively by 7.5% on reasoning with perturbations and 15.6% on compositional generalization tests.