# Semi-supervised segmentation of land cover images using nonlinear canonical correlation analysis with multiple features and t-SNE

arXiv: https://arxiv.org/abs/2401.12164 · HTML: https://mlsanity.com/paper/2401.12164/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 45 references that are in the map.
- First posted on arXiv 22 Jan 2024.

## Builds on (most cited first, 9 of 9)

- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives](https://mlsanity.com/paper/2208.07422/) (2022)
- [Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation](https://mlsanity.com/paper/1503.00848/) (2015)
- [Unsupervised Learning of Image Segmentation Based on Differentiable Feature Clustering](https://mlsanity.com/paper/2007.09990/) (2020)
- [Theoretical Foundations of t-SNE for Visualizing High-Dimensional Clustered Data](https://mlsanity.com/paper/2105.07536/) (2021)
- [Unsupervised Semantic Segmentation with Self-supervised Object-centric Representations](https://mlsanity.com/paper/2207.05027/) (2022)
- [Coarse-to-fine Semantic Segmentation from Image-level Labels](https://mlsanity.com/paper/1812.10885/) (2018)
- [Structured Sparse Subspace Clustering: A Joint Affinity Learning and Subspace Clustering Framework](https://mlsanity.com/paper/1610.05211/) (2016)
- [Deep Covariance Alignment for Domain Adaptive Remote Sensing Image Segmentation](https://mlsanity.com/paper/2401.04412/) (2024)

## Abstract (arXiv)

Image segmentation is a clustering task whereby each pixel is assigned a cluster label. Remote sensing data usually consists of multiple bands of spectral images in which there exist semantically meaningful land cover subregions, co-registered with other source data such as LIDAR (LIght Detection And Ranging) data, where available. This suggests that, in order to account for spatial correlation between pixels, a feature vector associated with each pixel may be a vectorized tensor representing the multiple bands and a local patch as appropriate. Similarly, multiple types of texture features based on a pixel's local patch would also be beneficial for encoding locally statistical information and spatial variations, without necessarily labelling pixel-wise a large amount of ground truth, then training a supervised model, which is sometimes impractical. In this work, by resorting to label only a small quantity of pixels, a new semi-supervised segmentation approach is proposed. Initially, over all pixels, an image data matrix is created in high dimensional feature space. Then, t-SNE projects the high dimensional data onto 3D embedding. By using radial basis functions as input features, which use the labelled data samples as centres, to pair with the output class labels, a modified canonical correlation analysis algorithm, referred to as RBF-CCA, is introduced which learns the associated projection matrix via the small labelled data set. The associated canonical variables, obtained for the full image, are applied by k-means clustering algorithm. The proposed semi-supervised RBF-CCA algorithm has been implemented on several remotely sensed multispectral images, demonstrating excellent segmentation results.