# Towards Understanding Ambiguity Resolution in Multimodal Inference of Meaning

arXiv: https://arxiv.org/abs/2510.09815 · HTML: https://mlsanity.com/paper/2510.09815/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 75 references that are in the map.
- First posted on arXiv 10 Oct 2025.

## Builds on (most cited first, 16 of 16)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Visually Grounded Reasoning across Languages and Cultures](https://mlsanity.com/paper/2109.13238/) (2021)
- [RedCaps: web-curated image-text data created by the people, for the people](https://mlsanity.com/paper/2111.11431/) (2021)
- [Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset](https://mlsanity.com/paper/2205.12522/) (2022)
- [xGQA: Cross-Lingual Visual Question Answering](https://mlsanity.com/paper/2109.06082/) (2021)
- [MURAL: Multimodal, Multitask Retrieval Across Languages](https://mlsanity.com/paper/2109.05125/) (2021)
- [MULE: Multimodal Universal Language Embedding](https://mlsanity.com/paper/1909.03493/) (2019)
- [Learning to Scale Multilingual Representations for Vision-Language Tasks](https://mlsanity.com/paper/2004.04312/) (2020)
- [Quantifying the Roles of Visual, Linguistic, and Visual-Linguistic Complexity in Verb Acquisition](https://mlsanity.com/paper/2304.02492/) (2023)

## Abstract (arXiv)

We investigate a new setting for foreign language learning, where learners infer the meaning of unfamiliar words in a multimodal context of a sentence describing a paired image. We conduct studies with human participants using different image-text pairs. We analyze the features of the data (i.e., images and texts) that make it easier for participants to infer the meaning of a masked or unfamiliar word, and what language backgrounds of the participants correlate with success. We find only some intuitive features have strong correlations with participant performance, prompting the need for further investigating of predictive features for success in these tasks. We also analyze the ability of AI systems to reason about participant performance, and discover promising future directions for improving this reasoning ability.