# Self-Supervised Learning to Guide Scientifically Relevant Categorization of Martian Terrain Images

arXiv: https://arxiv.org/abs/2204.09854 · HTML: https://mlsanity.com/paper/2204.09854/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 67 references that are in the map.
- First posted on arXiv 21 Apr 2022.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Colorization as a Proxy Task for Visual Understanding](https://mlsanity.com/paper/1703.04044/) (2017)
- [Deep TEN: Texture Encoding Network](https://mlsanity.com/paper/1612.02844/) (2016)
- [Deep Texture Manifold for Ground Terrain Recognition](https://mlsanity.com/paper/1803.10896/) (2018)
- [Improved Bilinear Pooling with CNNs](https://mlsanity.com/paper/1707.06772/) (2017)
- [Mars Image Content Classification: Three Years of NASA Deployment and Recent Advances](https://mlsanity.com/paper/2102.05011/) (2021)
- [DeepTerramechanics: Terrain Classification and Slip Estimation for Ground Robots via Deep Learning](https://mlsanity.com/paper/1806.07379/) (2018)
- [Multi-level Texture Encoding and Representation (MuLTER) based on Deep Neural Networks](https://mlsanity.com/paper/1905.09907/) (2019)
- [SPASS: Scientific Prominence Active Search System with Deep Image Captioning Network](https://mlsanity.com/paper/1809.03385/) (2018)

## Built on by (most cited first, 5 of 5)

- [Improving Contrastive Learning on Visually Homogeneous Mars Rover Images](https://mlsanity.com/paper/2210.09234/) (2022)
- [Enhancing Martian Terrain Recognition with Deep Constrained Clustering](https://mlsanity.com/paper/2503.17633/) (2025)
- [Mixed-domain Training Improves Multi-Mission Terrain Segmentation](https://mlsanity.com/paper/2209.13674/) (2022)
- [Semi-Supervised Representative Region Texture Extraction of Faccade](https://mlsanity.com/paper/2212.02220/) (2022)
- [M3Depth: Wavelet-Enhanced Depth Estimation on Mars via Mutual Boosting of Dual-Modal Data](https://mlsanity.com/paper/2505.14159/) (2025)

## Abstract (arXiv)

Automatic terrain recognition in Mars rover images is an important problem not just for navigation, but for scientists interested in studying rock types, and by extension, conditions of the ancient Martian paleoclimate and habitability. Existing approaches to label Martian terrain either involve the use of non-expert annotators producing taxonomies of limited granularity (e.g. soil, sand, bedrock, float rock, etc.), or rely on generic class discovery approaches that tend to produce perceptual classes such as rover parts and landscape, which are irrelevant to geologic analysis. Expert-labeled datasets containing granular geological/geomorphological terrain categories are rare or inaccessible to public, and sometimes require the extraction of relevant categorical information from complex annotations. In order to facilitate the creation of a dataset with detailed terrain categories, we present a self-supervised method that can cluster sedimentary textures in images captured from the Mast camera onboard the Curiosity rover (Mars Science Laboratory). We then present a qualitative analysis of these clusters and describe their geologic significance via the creation of a set of granular terrain categories. The precision and geologic validation of these automatically discovered clusters suggest that our methods are promising for the rapid classification of important geologic features and will therefore facilitate our long-term goal of producing a large, granular, and publicly available dataset for Mars terrain recognition.