# Learning Multimodal Word Representation via Dynamic Fusion Methods

arXiv: https://arxiv.org/abs/1801.00532 · HTML: https://mlsanity.com/paper/1801.00532/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jan 2018.

## Built on by (most cited first, 7 of 7)

- [Play to the Score: Stage-Guided Dynamic Multi-Sensory Fusion for Robotic Manipulation](https://mlsanity.com/paper/2408.01366/) (2024)
- [TokAlign: Efficient Vocabulary Adaptation via Token Alignment](https://mlsanity.com/paper/2506.03523/) (2025)
- [Looking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling](https://mlsanity.com/paper/2510.08470/) (2025)
- [Language Cognition and Language Computation -- Human and Machine Language Understanding](https://mlsanity.com/paper/2301.04788/) (2023)
- [A Brain-inspired Computational Model for Human-like Concept Learning](https://mlsanity.com/paper/2401.06471/) (2024)
- [TokAlign++: Advancing Vocabulary Adaptation via Better Token Alignment](https://mlsanity.com/paper/2605.13429/) (2026)
- [Learning grounded word meaning representations on similarity graphs](https://mlsanity.com/paper/2109.03084/) (2021)

## Abstract (arXiv)

Multimodal models have been proven to outperform text-based models on learning semantic word representations. Almost all previous multimodal models typically treat the representations from different modalities equally. However, it is obvious that information from different modalities contributes differently to the meaning of words. This motivates us to build a multimodal model that can dynamically fuse the semantic representations from different modalities according to different types of words. To that end, we propose three novel dynamic fusion methods to assign importance weights to each modality, in which weights are learned under the weak supervision of word association pairs. The extensive experiments have demonstrated that the proposed methods outperform strong unimodal baselines and state-of-the-art multimodal models.