mlsanityData to 7 Oct 2026

Contrastive ground-level image and remote sensing pre-training improves representation learning for natural world imagery

arXiv 2409.19439 ↗First posted 28 Sep 2024cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Multimodal image-text contrastive learning has shown that joint representations can be learned across modalities. Here, we show how leveraging multiple views of image data with contrastive learning can improve downstream fine-grained classification performance for species recognition, even when one view is absent. We propose ContRastive Image-remote Sensing Pre-training (CRISP)unicodex2014a new pre-training task for ground-level and aerial image representation learning of the natural worldunicodex2014and introduce Nature Multi-View (NMV), a dataset of natural world imagery including >3 million ground-level and aerial image pairs for over 6,000 plant taxa across the ecologically diverse state of California. The NMV dataset and accompanying material are available at hf.co/datasets/andyvhuynh/NatureMultiView.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 28 Sep 2024.

Builds on

32 of 50 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  6. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  7. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  8. 2021Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Datacited by 214
  9. 2020Geography-Aware Self-Supervised Learningcited by 190
  10. 2020VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrievalcited by 188
  11. 2015Wide-Area Image Geolocalization with Aerial Reference Imagerycited by 185
  12. 2023GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localizationcited by 185
20 more
  1. 2019Lending Orientation to Neural Networks for Cross-view Geo-localizationcited by 179
  2. 2021Benchmarking Representation Learning for Natural World Image Collectionscited by 167
  3. 2022SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understandingcited by 164
  4. 2020University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localizationcited by 152
  5. 2021Seeing biodiversity: perspectives in machine learning for wildlife conservationcited by 129
  6. 2023GEO-Bench: Toward Foundation Models for Earth Monitoringcited by 122
  7. 2018Tile2Vec: Unsupervised representation learning for spatially distributed datacited by 109
  8. 2016Localizing and Orienting Street Views Using Overhead Imagerycited by 93
  9. 2021When Does Contrastive Visual Representation Learning Work?cited by 92
  10. 2017The Devil is in the Tails: Fine-grained Classification in the Wildcited by 89
  11. 2021Extending the WILDS Benchmark for Unsupervised Adaptationcited by 85
  12. 2023Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignmentcited by 71
  13. 2023CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representationscited by 67
  14. 2023PIGEON: Predicting Image Geolocationscited by 55
  15. 2021Geographical Knowledge-driven Representation Learning for Remote Sensing Imagescited by 37
  16. 2021Focus on the Positives: Self-Supervised Learning for Biodiversity Monitoringcited by 16
  17. 2023OpenForest: A data catalogue for machine learning in forest monitoringcited by 15
  18. 2021Representation Learning for Remote Sensing: An Unsupervised Sensor Fusion Approachcited by 14
  19. 2023BirdSAT: Cross-View Contrastive Masked Autoencoders for Bird Species Classification and Mappingcited by 12
  20. 2023SatBird: Bird Species Distribution Modeling with Remote Sensing and Citizen Science Datacited by 3

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 6.

10 papers.

  1. 2025TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysiscited by 35
  2. 2024TaxaBind: A Unified Embedding Space for Ecological Applicationscited by 33
  3. 2025RANGE: Retrieval Augmented Neural Fields for Multi-Resolution Geo-Embeddingscited by 18
  4. 2024Video2BEV: Transforming Drone Videos to BEVs for Video-based Geo-localizationcited by 16
  5. 2024WildSAT: Learning Satellite Image Representations from Wildlife Observationscited by 13
  6. 2025GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representationscited by 9
  7. 2024DivShift: Exploring Domain-Specific Distribution Shifts in Large-Scale, Volunteer-Collected Biodiversity Datasetscited by 4
  8. 2025Global and Local Entailment Learning for Natural World Imagerycited by 3
  9. 2025ProM3E: Probabilistic Masked MultiModal Embedding Model for Ecologycited by 1
  10. 2025SatDINO: A Deep Dive into Self-Supervised Pretraining for Remote Sensingcited by 0