# SSL-SoilNet: A Hybrid Transformer-based Framework with Self-Supervised Learning for Large-scale Soil Organic Carbon Prediction

arXiv: https://arxiv.org/abs/2308.03586 · HTML: https://mlsanity.com/paper/2308.03586/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 76 references that are in the map.
- First posted on arXiv 7 Aug 2023.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Deep Forest](https://mlsanity.com/paper/1702.08835/) (2017)
- [TeleViT: Teleconnection-driven Transformers Improve Subseasonal to Seasonal Wildfire Forecasting](https://mlsanity.com/paper/2306.10940/) (2023)
- [Predicting and Mapping of Soil Organic Carbon Using Machine Learning Algorithms in Northern Iran](https://mlsanity.com/paper/2007.12475/) (2020)
- [Visual Transformer for Soil Classification](https://mlsanity.com/paper/2209.02950/) (2022)

## Built on by (most cited first, 5 of 5)

- [A Data-Driven Review of Remote Sensing-Based Data Fusion in Precision Agriculture from Foundational to Transformer-Based Techniques](https://mlsanity.com/paper/2410.18353/) (2024)
- [Fusion of Deep Learning and GIS for Advanced Remote Sensing Image Analysis](https://mlsanity.com/paper/2412.19856/) (2024)
- [Fine-Scale Soil Mapping in Alaska with Multimodal Machine Learning](https://mlsanity.com/paper/2506.17302/) (2025)
- [A study of Universal ODE approaches to predicting soil organic carbon](https://mlsanity.com/paper/2509.24306/) (2025)
- [Spatial Prediction of Soil Microplastics and Organic Matter Using Graph Attention Networks](https://mlsanity.com/paper/2607.22875/) (2026)

## Abstract (arXiv)

Soil Organic Carbon (SOC) constitutes a fundamental component of terrestrial ecosystem functionality, playing a pivotal role in nutrient cycling, hydrological balance, and erosion mitigation. Precise mapping of SOC distribution is imperative for the quantification of ecosystem services, notably carbon sequestration and soil fertility enhancement. Digital soil mapping (DSM) leverages statistical models and advanced technologies, including machine learning (ML), to accurately map soil properties, such as SOC, utilizing diverse data sources like satellite imagery, topography, remote sensing indices, and climate series. Within the domain of ML, self-supervised learning (SSL), which exploits unlabeled data, has gained prominence in recent years. This study introduces a novel approach that aims to learn the geographical link between multimodal features via self-supervised contrastive learning, employing pretrained Vision Transformers (ViT) for image inputs and Transformers for climate data, before fine-tuning the model with ground reference samples. The proposed approach has undergone rigorous testing on two distinct large-scale datasets, with results indicating its superiority over traditional supervised learning models, which depends solely on labeled data. Furthermore, through the utilization of various evaluation metrics (e.g., RMSE, MAE, CCC, etc.), the proposed model exhibits higher accuracy when compared to other conventional ML algorithms like random forest and gradient boosting. This model is a robust tool for predicting SOC and contributes to the advancement of DSM techniques, thereby facilitating land management and decision-making processes based on accurate information.