# Datacube segmentation via Deep Spectral Clustering

arXiv: https://arxiv.org/abs/2401.17695 · HTML: https://mlsanity.com/paper/2401.17695/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 115 references that are in the map.
- First posted on arXiv 31 Jan 2024.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering](https://mlsanity.com/paper/1611.05148/) (2016)
- [A survey of loss functions for semantic segmentation](https://mlsanity.com/paper/2006.14822/) (2020)
- [Joint Unsupervised Learning of Deep Representations and Image Clusters](https://mlsanity.com/paper/1604.03628/) (2016)
- [Variational Lossy Autoencoder](https://mlsanity.com/paper/1611.02731/) (2016)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation](https://mlsanity.com/paper/1810.07842/) (2018)
- [InfoVAE: Information Maximizing Variational Autoencoders](https://mlsanity.com/paper/1706.02262/) (2017)
- [Rotation-invariant convolutional neural networks for galaxy morphology prediction](https://mlsanity.com/paper/1503.07077/) (2015)
- [A Living Review of Machine Learning for Particle Physics](https://mlsanity.com/paper/2102.02770/) (2021)
- [Manifold learning: what, how, and why](https://mlsanity.com/paper/2311.03757/) (2023)
- [Clustering with Deep Learning: Taxonomy and New Methods](https://mlsanity.com/paper/1801.07648/) (2018)
- [Loss Functions in the Era of Semantic Segmentation: A Survey and Outlook](https://mlsanity.com/paper/2312.05391/) (2023)
- [Deep Continuous Clustering](https://mlsanity.com/paper/1803.01449/) (2018)
- [ExoMiner: A Highly Accurate and Explainable Deep Learning Classifier that Validates 301 New Exoplanets](https://mlsanity.com/paper/2111.10009/) (2021)
- [Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy](https://mlsanity.com/paper/2211.03796/) (2022)
- [DeepCMB: Lensing Reconstruction of the Cosmic Microwave Background with Deep Neural Networks](https://mlsanity.com/paper/1810.01483/) (2018)
- [Recovering the number of clusters in data sets with noise features using feature rescaling factors](https://mlsanity.com/paper/1602.06989/) (2016)
- [The Mathematical Foundations of Manifold Learning](https://mlsanity.com/paper/2011.01307/) (2020)
- [Can Deep Learning Assist Automatic Identification of Layered Pigments From XRF Data?](https://mlsanity.com/paper/2207.12651/) (2022)
- [Identifying Exoplanets with Machine Learning Methods: A Preliminary Study](https://mlsanity.com/paper/2204.00721/) (2022)
- [ganX -- generate artificially new XRF a python library to generate MA-XRF raw data out of RGB images](https://mlsanity.com/paper/2304.14078/) (2023)

## Abstract (arXiv)

Extended Vision techniques are ubiquitous in physics. However, the data cubes steaming from such analysis often pose a challenge in their interpretation, due to the intrinsic difficulty in discerning the relevant information from the spectra composing the data cube. Furthermore, the huge dimensionality of data cube spectra poses a complex task in its statistical interpretation; nevertheless, this complexity contains a massive amount of statistical information that can be exploited in an unsupervised manner to outline some essential properties of the case study at hand, e.g.~it is possible to obtain an image segmentation via (deep) clustering of data-cube's spectra, performed in a suitably defined low-dimensional embedding space. To tackle this topic, we explore the possibility of applying unsupervised clustering methods in encoded space, i.e. perform deep clustering on the spectral properties of datacube pixels. A statistical dimensional reduction is performed by an ad hoc trained (Variational) AutoEncoder, in charge of mapping spectra into lower dimensional metric spaces, while the clustering process is performed by a (learnable) iterative K-Means clustering algorithm. We apply this technique to two different use cases, of different physical origins: a set of Macro mapping X-Ray Fluorescence (MA-XRF) synthetic data on pictorial artworks, and a dataset of simulated astrophysical observations.