mlsanityData to 7 Oct 2026

On Deep Multi-View Representation Learning: Objectives and Optimization

arXiv 1602.01024 ↗First posted 2 Feb 2016cs.LGcited by 162

Not on the map. Why?

Abstract (arXiv)

We consider learning representations (features) in the setting in which we have access to multiple unlabeled views of the data for learning while only one view is available for downstream tasks. Previous work on this problem has proposed several techniques based on deep neural networks, typically involving either autoencoder-like networks with a reconstruction objective or paired feedforward networks with a batch-style correlation-based objective. We analyze several techniques based on prior work, as well as new variants, and compare them empirically on image, speech, and text tasks. We find an advantage for correlation-based representation learning, while the best results on most tasks are obtained with our new variant, deep canonically correlated autoencoders (DCCAE). We also explore a stochastic optimization procedure for minibatch correlation-based objectives and discuss the time/performance trade-offs for kernel-based and neural network-based implementations.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 30; 2023: 34; 2024: 27; 2025: 24; 2026: 21.

162 papers.

Sort:

By area:

  1. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  2. 2023A Cookbook of Self-Supervised Learningcited by 248
  3. 2022Trusted Multi-View Classification with Dynamic Evidential Fusioncited by 167
  4. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  5. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  6. 2021Trusted Multi-View Classificationcited by 94
  7. 2022GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversationcited by 86
  8. 2022Deep Clustering: A Comprehensive Surveycited by 78
  9. 2023Multimodal Representation Learning by Alternating Unimodal Adaptationcited by 74
  10. 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
  11. 2023Self-Supervised Multimodal Learning: A Surveycited by 58
  12. 2021Reconsidering Representation Alignment for Multi-view Clusteringcited by 57
38 more
  1. 2024Reliable Conflictive Multi-View Learningcited by 45
  2. 2021This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagationcited by 43
  3. 2021A Variational Information Bottleneck Approach to Multi-Omics Data Integrationcited by 40
  4. 2021Understanding Latent Correlation-Based Multiview Learning and Self-Supervision: An Identifiability Perspectivecited by 37
  5. 2021Multimodal Federated Learning on IoT Datacited by 32
  6. 2024Coupled Mamba: Enhanced Multi-modal Fusion with Coupled State Space Modelcited by 30
  7. 2022Uncertainty-Aware Multi-View Representation Learningcited by 29
  8. 2025Understanding the Emergence of Multimodal Representation Alignmentcited by 28
  9. 2023Decoupled Contrastive Multi-View Clustering with High-Order Random Walkscited by 23
  10. 2023Incomplete Multi-view Clustering via Prototype-based Imputationcited by 21
  11. 2023DiME: Maximizing Mutual Information by a Difference of Matrix-Based Entropiescited by 20
  12. 2022Robust and Explainable Autoencoders for Unsupervised Time Series Outlier Detection---Extended Versioncited by 19
  13. 2022Contrastive Multi-view Hyperbolic Hierarchical Clusteringcited by 19
  14. 2021Deep Co-Attention Network for Multi-View Subspace Learningcited by 17
  15. 2024AlignMamba: Enhancing Multimodal Mamba with Local and Global Cross-modal Alignmentcited by 17
  16. 2021Heterogeneous Contrastive Learningcited by 15
  17. 2024Trusted Unified Feature-Neighborhood Dynamics for Multi-View Classificationcited by 15
  18. 2022Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learningcited by 13
  19. 2023A Reliable and Interpretable Framework of Multi-view Learning for Liver Fibrosis Stagingcited by 13
  20. 2023Multimodal Federated Learning in Healthcare: a Reviewcited by 13
  21. 2024SDR-GNN: Spectral Domain Reconstruction Graph Neural Network for Incomplete Multimodal Learning in Conversational Emotion Recognitioncited by 13
  22. 2021RELAX: Representation Learning Explainabilitycited by 12
  23. 2022Gacs-Korner Common Information Variational Autoencodercited by 12
  24. 2023Semantic Invariant Multi-view Clustering with Fully Incomplete Informationcited by 12
  25. 2024MERIT: Multi-view evidential learning for reliable and interpretable liver fibrosis stagingcited by 12
  26. 2021ASMR: Learning Attribute-Based Person Search with Adaptive Semantic Margin Regularizercited by 11
  27. 2021Continual learning in cross-modal retrievalcited by 10
  28. 2022UPAR: Unified Pedestrian Attribute Recognition and Person Retrievalcited by 10
  29. 2022Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picturecited by 10
  30. 2023On the Effects of Self-supervision and Contrastive Alignment in Deep Multi-view Clusteringcited by 10
  31. 2023Max-Sliced Mutual Informationcited by 10
  32. 2022Latent Heterogeneous Graph Network for Incomplete Multi-View Learningcited by 9
  33. 2021Disentangled Variational Information Bottleneck for Multiview Representation Learningcited by 8
  34. 2022Consensus Learning for Cooperative Multi-Agent Reinforcement Learningcited by 8
  35. 2024Trusted Multi-view Learning under Noisy Supervisioncited by 8
  36. 2025Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecastingcited by 8
  37. 2021An Online Riemannian PCA for Stochastic Canonical Correlation Analysiscited by 7
  38. 2025Deep Incomplete Multi-view Learning via Cyclic Permutation of VAEscited by 7

50 most cited of 162 All 162 (JSON)