# Learning Subspace-Preserving Sparse Attention Graphs from Heterogeneous Multiview Data

arXiv: https://arxiv.org/abs/2605.11881 · HTML: https://mlsanity.com/paper/2605.11881/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Sparse Sequence-to-Sequence Models](https://mlsanity.com/paper/1905.05702/) (2019)
- [LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collections](https://mlsanity.com/paper/2305.18287/) (2023)
- [Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP](https://mlsanity.com/paper/2310.00927/) (2023)
- [Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models](https://mlsanity.com/paper/2306.05272/) (2023)
- [MIM-Refiner: A Contrastive Learning Boost from Intermediate Pre-Trained Representations](https://mlsanity.com/paper/2402.10093/) (2024)
- [Let Go of Your Labels with Unsupervised Transfer](https://mlsanity.com/paper/2406.07236/) (2024)
- [PRO-VPT: Distribution-Adaptive Visual Prompt Tuning via Prompt Relocation](https://mlsanity.com/paper/2503.06901/) (2025)
- [THESAURUS: Contrastive Graph Clustering by Swapping Fused Gromov-Wasserstein Couplings](https://mlsanity.com/paper/2412.11550/) (2024)
- [Hierarchical Sparse Representation Clustering for High-Dimensional Data Streams](https://mlsanity.com/paper/2409.04698/) (2024)
- [A General Representation-Based Approach to Multi-Source Domain Adaptation](https://mlsanity.com/paper/2604.23790/) (2026)
- [Multiview Self-Representation Learning across Heterogeneous Views](https://mlsanity.com/paper/2602.04328/) (2026)

## Abstract (arXiv)

The high-dimensional features extracted from large-scale unlabeled data via various pretrained models with diverse architectures are referred to as heterogeneous multiview data. Most existing unsupervised transfer learning methods fail to faithfully recover intrinsic subspace structures when exploiting complementary information across multiple views. Therefore, a fundamental challenge involves constructing sparse similarity graphs that preserve these underlying subspace structures for achieving semantic alignment across heterogeneous views. In this paper, we propose a sparse attention graph learning (SAGL) method that learns subspace-preserving sparse attention graphs from heterogeneous multiview data. Specifically, we introduce a bilinear attention factorization scheme to capture asymmetric similarities among the high-dimensional features, which breaks the symmetry bottleneck that is inherent in the traditional representation learning techniques. A dynamic sparsity gating mechanism then predicts a feature-specific compression factor for adaptively controlling the topological contributions of neighbors. Furthermore, we employ a structured sparse projection via α-entmax to generate subspace-preserving sparse attention graphs for individual views. SAGL leverages these view-specific graphs to conduct sparse information aggregation, yielding discriminative representations for multiview learning tasks. In addition, we provide a rigorous theoretical analysis that bridges differentiable sparse attention and probability simplex constraints. Extensive experiments conducted on multiple benchmark datasets demonstrate that SAGL consistently outperforms the state-of-the-art unsupervised transfer learning approaches.