mlsanityData to 14 Oct 2026

MarsRecon: Self-Supervised and Multimodal Surface Representations for Mars

arXiv 2609.22379 ↗First posted 17 Sep 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

High-resolution orbital imagery offers a rich record of the Martian surface, but sparse geological labels limit supervised representation learning. We present MarsRecon, a geospatially aware pipeline for learning visual and multimodal representations from HiRISE observations of Olympus Mons. The pipeline calibrates NASA Planetary Data System products, extracts valid georeferenced patches, and trains a masked autoencoder on unlabeled imagery. Increasing input resolution and filtering invalid tokens reduced held-out reconstruction loss from 0.1751 to 0.1342 in the principal Stage A model series. We then freeze the visual encoder and align its features with observation text, coordinates, and local--global image context. The strongest current local-primary model achieves image-to-text recall@10 of 0.3787, text-to-image recall@10 of 0.9161, and local-to-global recall@10 of 0.4350 on the held-out test split. These results establish a working Mars-specific pretraining and retrieval pipeline; further crop-overlap controls and downstream geological evaluations are needed to assess the broader utility of its embeddings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 24 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

13 of 24 references.

References · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  7. 2018On the Spectral Bias of Neural Networkscited by 1,341
  8. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 408
  9. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 363
  10. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 308
  11. 2020Focal Frequency Loss for Image Reconstruction and Synthesiscited by 206
  12. 2023GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localizationcited by 187
1 more
  1. 2021TorchGeo: Deep Learning With Geospatial Datacited by 73

Built on by

Not cited yet.