# Dynamic Multimodal Fusion via Meta-Learning Towards Micro-Video Recommendation

arXiv: https://arxiv.org/abs/2501.07110 · HTML: https://mlsanity.com/paper/2501.07110/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 13 Jan 2025.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [Multimodal Deep Learning](https://mlsanity.com/paper/2301.04856/) (2023)
- [VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback](https://mlsanity.com/paper/1510.01784/) (2015)
- [MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation](https://mlsanity.com/paper/1908.00413/) (2019)
- [ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation](https://mlsanity.com/paper/1905.10089/) (2019)
- [Mining Latent Structures for Multimedia Recommendation](https://mlsanity.com/paper/2104.09036/) (2021)
- [GRCN: Graph-Refined Convolutional Network for Multimedia Recommendation with Implicit Feedback](https://mlsanity.com/paper/2111.02036/) (2021)
- [Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition](https://mlsanity.com/paper/2010.09982/) (2020)
- [Interest-aware Message-Passing GCN for Recommendation](https://mlsanity.com/paper/2102.10044/) (2021)
- [Deep Multimodal Learning for Audio-Visual Speech Recognition](https://mlsanity.com/paper/1501.05396/) (2015)
- [Learning to Warm Up Cold Item Embeddings for Cold-start Recommendation with Meta Scaling and Shifting Networks](https://mlsanity.com/paper/2105.04790/) (2021)
- [Disentangled Multimodal Representation Learning for Recommendation](https://mlsanity.com/paper/2203.05406/) (2022)
- [Attentive Long Short-Term Preference Modeling for Personalized Product Search](https://mlsanity.com/paper/1811.10155/) (2018)
- [User Diverse Preference Modeling by Multimodal Attentive Metric Learning](https://mlsanity.com/paper/1908.07738/) (2019)
- [HS-GCN: Hamming Spatial Graph Convolutional Networks for Recommendation](https://mlsanity.com/paper/2301.05430/) (2023)
- [Discrete Factorization Machines for Fast Feature-based Recommendation](https://mlsanity.com/paper/1805.02232/) (2018)

## Built on by (most cited first, 4 of 4)

- [Do Recommender Systems Really Leverage Multimodal Content? A Comprehensive Analysis on Multimodal Representations for Recommendation](https://mlsanity.com/paper/2508.04571/) (2025)
- [Multimodal Representation Learning Conditioned on Semantic Relations](https://mlsanity.com/paper/2508.17497/) (2025)
- [TimeMM: Time-as-Operator Spectral Filtering for Dynamic Multimodal Recommendation](https://mlsanity.com/paper/2604.26247/) (2026)
- [Preference Flow Matching with Spectral Factorization for Micro-video Recommendation](https://mlsanity.com/paper/2608.26579/) (2026)

## Abstract (arXiv)

Multimodal information (e.g., visual, acoustic, and textual) has been widely used to enhance representation learning for micro-video recommendation. For integrating multimodal information into a joint representation of micro-video, multimodal fusion plays a vital role in the existing micro-video recommendation approaches. However, the static multimodal fusion used in previous studies is insufficient to model the various relationships among multimodal information of different micro-videos. In this paper, we develop a novel meta-learning-based multimodal fusion framework called Meta Multimodal Fusion (MetaMMF), which dynamically assigns parameters to the multimodal fusion function for each micro-video during its representation learning. Specifically, MetaMMF regards the multimodal fusion of each micro-video as an independent task. Based on the meta information extracted from the multimodal features of the input task, MetaMMF parameterizes a neural network as the item-specific fusion function via a meta learner. We perform extensive experiments on three benchmark datasets, demonstrating the significant improvements over several state-of-the-art multimodal recommendation models, like MMGCN, LATTICE, and InvRL. Furthermore, we lighten our model by adopting canonical polyadic decomposition to improve the training efficiency, and validate its effectiveness through experimental results. Codes are available at https://github.com/hanliu95/MetaMMF.