# MetaViewer: Towards A Unified Multi-View Representation

arXiv: https://arxiv.org/abs/2303.06329 · HTML: https://mlsanity.com/paper/2303.06329/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 78 references that are in the map.
- First posted on arXiv 11 Mar 2023.

## Builds on (most cited first, 30 of 33)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Convolutional Two-Stream Network Fusion for Video Action Recognition](https://mlsanity.com/paper/1604.06573/) (2016)
- [MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning](https://mlsanity.com/paper/1903.10258/) (2019)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [On Deep Multi-View Representation Learning: Objectives and Optimization](https://mlsanity.com/paper/1602.01024/) (2016)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Learned Optimizers that Scale and Generalize](https://mlsanity.com/paper/1703.04813/) (2017)
- [A Survey of Multi-View Representation Learning](https://mlsanity.com/paper/1610.01206/) (2016)
- [Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation](https://mlsanity.com/paper/1910.13616/) (2019)
- [A Survey of Deep Meta-Learning](https://mlsanity.com/paper/2010.03522/) (2020)
- [SiT: Self-supervised vIsion Transformer](https://mlsanity.com/paper/2104.03602/) (2021)
- [Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers](https://mlsanity.com/paper/2103.16553/) (2021)
- [Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis](https://mlsanity.com/paper/1911.05544/) (2019)
- [Multi-level Feature Learning for Contrastive Multi-view Clustering](https://mlsanity.com/paper/2106.11193/) (2021)
- [Self-Supervised Discriminative Feature Learning for Deep Multi-View Clustering](https://mlsanity.com/paper/2103.15069/) (2021)
- [Towards Discriminative Representation: Multi-view Trajectory Contrastive Learning for Online Multi-object Tracking](https://mlsanity.com/paper/2203.14208/) (2022)
- [Self-Filtering: A Noise-Aware Sample Selection for Label Noise with Confidence Penalization](https://mlsanity.com/paper/2208.11351/) (2022)
- [Uncertainty-Aware Multi-View Representation Learning](https://mlsanity.com/paper/2201.05776/) (2022)
- [Multi-view Deep Subspace Clustering Networks](https://mlsanity.com/paper/1908.01978/) (2019)
- [Multi-View Spectral Clustering Tailored Tensor Low-Rank Representation](https://mlsanity.com/paper/2004.14705/) (2020)
- [Symbolic Learning to Optimize: Towards Interpretability and Scalability](https://mlsanity.com/paper/2203.06578/) (2022)
- [Self-Supervised Information Bottleneck for Deep Multi-View Subspace Clustering](https://mlsanity.com/paper/2204.12496/) (2022)
- [Fine-Grained Classification with Noisy Labels](https://mlsanity.com/paper/2303.02404/) (2023)
- [Learning to Rectify for Robust Learning with Noisy Labels](https://mlsanity.com/paper/2111.04239/) (2021)

## Built on by (most cited first, 5 of 5)

- [Generalizable and Robust Spectral Method for Multi-view Representation Learning](https://mlsanity.com/paper/2411.02138/) (2024)
- [Efficient and Effective Deep Multi-view Subspace Clustering](https://mlsanity.com/paper/2310.09718/) (2023)
- [Robust Fuzzy Multi-view Learning under View Conflict](https://mlsanity.com/paper/2605.24475/) (2026)
- [Pretraining Reusable Inference Across Views with Synthetic Task Priors](https://mlsanity.com/paper/2608.19115/) (2026)
- [Adversarial Consistency-Guided Representation Learning for Multi-view Clustering](https://mlsanity.com/paper/2609.35212/) (2026)

## Abstract (arXiv)

Existing multi-view representation learning methods typically follow a specific-to-uniform pipeline, extracting latent features from each view and then fusing or aligning them to obtain the unified object representation. However, the manually pre-specify fusion functions and view-private redundant information mixed in features potentially degrade the quality of the derived representation. To overcome them, we propose a novel bi-level-optimization-based multi-view learning framework, where the representation is learned in a uniform-to-specific manner. Specifically, we train a meta-learner, namely MetaViewer, to learn fusion and model the view-shared meta representation in outer-level optimization. Start with this meta representation, view-specific base-learners are then required to rapidly reconstruct the corresponding view in inner-level. MetaViewer eventually updates by observing reconstruction processes from uniform to specific over all views, and learns an optimal fusion scheme that separates and filters out view-private information. Extensive experimental results in downstream tasks such as classification and clustering demonstrate the effectiveness of our method.