# Cross-Modal Alignment Learning of Vision-Language Conceptual Systems

arXiv: https://arxiv.org/abs/2208.01744 · HTML: https://mlsanity.com/paper/2208.01744/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 31 Jul 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Connecting Vision and Language with Localized Narratives](https://mlsanity.com/paper/1912.03098/) (2019)
- [Rethinking Diversified and Discriminative Proposal Generation for Visual Grounding](https://mlsanity.com/paper/1805.03508/) (2018)
- [Learning Cross-modal Context Graph for Visual Grounding](https://mlsanity.com/paper/1911.09042/) (2019)
- [Weakly-supervised Visual Grounding of Phrases with Linguistic Structures](https://mlsanity.com/paper/1705.01371/) (2017)
- [Learning Aligned Cross-Modal Representations from Weakly Aligned Data](https://mlsanity.com/paper/1607.07295/) (2016)
- [Learning as the Unsupervised Alignment of Conceptual Systems](https://mlsanity.com/paper/1906.09012/) (2019)
- [Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? -- A computational investigation](https://mlsanity.com/paper/2109.14200/) (2021)
- [Message Passing Adaptive Resonance Theory for Online Active Semi-supervised Learning](https://mlsanity.com/paper/2012.01227/) (2020)

## Built on by (most cited first, 4 of 4)

- [Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment](https://mlsanity.com/paper/2212.10549/) (2022)
- [Semantic Composition in Visually Grounded Language Models](https://mlsanity.com/paper/2305.16328/) (2023)
- [The Ungrounded Alignment Problem](https://mlsanity.com/paper/2408.04242/) (2024)
- [Transformer-Driven Triple Fusion Framework for Enhanced Multimodal Author Intent Classification in Low-Resource Bangla](https://mlsanity.com/paper/2511.23287/) (2025)

## Abstract (arXiv)

Human infants learn the names of objects and develop their own conceptual systems without explicit supervision. In this study, we propose methods for learning aligned vision-language conceptual systems inspired by infants' word learning mechanisms. The proposed model learns the associations of visual objects and words online and gradually constructs cross-modal relational graph networks. Additionally, we also propose an aligned cross-modal representation learning method that learns semantic representations of visual objects and words in a self-supervised manner based on the cross-modal relational graph networks. It allows entities of different modalities with conceptually the same meaning to have similar semantic representation vectors. We quantitatively and qualitatively evaluate our method, including object-to-word mapping and zero-shot learning tasks, showing that the proposed model significantly outperforms the baselines and that each conceptual system is topologically aligned.