# Learning Structure Aware Deep Spectral Embedding

arXiv: https://arxiv.org/abs/2305.08215 · HTML: https://mlsanity.com/paper/2305.08215/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 92 references that are in the map.
- First posted on arXiv 14 May 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [Contrastive Clustering](https://mlsanity.com/paper/2009.09687/) (2020)
- [Joint Unsupervised Learning of Deep Representations and Image Clusters](https://mlsanity.com/paper/1604.03628/) (2016)
- [Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization](https://mlsanity.com/paper/1704.06327/) (2017)
- [SPICE: Semantic Pseudo-labeling for Image Clustering](https://mlsanity.com/paper/2103.09382/) (2021)
- [Deep Subspace Clustering Networks](https://mlsanity.com/paper/1709.02508/) (2017)
- [SpectralNet: Spectral Clustering using Deep Neural Networks](https://mlsanity.com/paper/1801.01587/) (2018)
- [Deep Comprehensive Correlation Mining for Image Clustering](https://mlsanity.com/paper/1904.06925/) (2019)
- [Twin Contrastive Learning for Online Clustering](https://mlsanity.com/paper/2210.11680/) (2022)
- [Scalable Sparse Subspace Clustering by Orthogonal Matching Pursuit](https://mlsanity.com/paper/1507.01238/) (2015)
- [Robust and Efficient Subspace Segmentation via Least Squares Regression](https://mlsanity.com/paper/1404.6736/) (2014)
- [Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering](https://mlsanity.com/paper/1605.02633/) (2016)
- [Subspace Clustering by Block Diagonal Representation](https://mlsanity.com/paper/1805.09243/) (2018)
- [Self-Supervised Convolutional Subspace Clustering Network](https://mlsanity.com/paper/1905.00149/) (2019)
- [Structured Graph Learning for Scalable Subspace Clustering: From Single-view to Multi-view](https://mlsanity.com/paper/2102.07943/) (2021)
- [Pseudo-supervised Deep Subspace Clustering](https://mlsanity.com/paper/2104.03531/) (2021)
- [Deep Discriminative Clustering Analysis](https://mlsanity.com/paper/1905.01681/) (2019)
- [Learning a Self-Expressive Network for Subspace Clustering](https://mlsanity.com/paper/2110.04318/) (2021)
- [Neural Collaborative Subspace Clustering](https://mlsanity.com/paper/1904.10596/) (2019)
- [Stochastic Sparse Subspace Clustering](https://mlsanity.com/paper/2005.01449/) (2020)
- [Towards Clustering-friendly Representations: Subspace Clustering via Graph Filtering](https://mlsanity.com/paper/2106.09874/) (2021)
- [Maximum Entropy Subspace Clustering Network](https://mlsanity.com/paper/2012.03176/) (2020)
- [Scalable and Robust Sparse Subspace Clustering Using Randomized Clustering and Multilayer Graphs](https://mlsanity.com/paper/1802.07648/) (2018)
- [Relation-Guided Representation Learning](https://mlsanity.com/paper/2007.05742/) (2020)
- [Overcomplete Deep Subspace Clustering Networks](https://mlsanity.com/paper/2011.08306/) (2020)

## Abstract (arXiv)

Spectral Embedding (SE) has often been used to map data points from non-linear manifolds to linear subspaces for the purpose of classification and clustering. Despite significant advantages, the subspace structure of data in the original space is not preserved in the embedding space. To address this issue subspace clustering has been proposed by replacing the SE graph affinity with a self-expression matrix. It works well if the data lies in a union of linear subspaces however, the performance may degrade in real-world applications where data often spans non-linear manifolds. To address this problem we propose a novel structure-aware deep spectral embedding by combining a spectral embedding loss and a structure preservation loss. To this end, a deep neural network architecture is proposed that simultaneously encodes both types of information and aims to generate structure-aware spectral embedding. The subspace structure of the input data is encoded by using attention-based self-expression learning. The proposed algorithm is evaluated on six publicly available real-world datasets. The results demonstrate the excellent clustering performance of the proposed algorithm compared to the existing state-of-the-art methods. The proposed algorithm has also exhibited better generalization to unseen data points and it is scalable to larger datasets without requiring significant computational resources.