# TaxaBind: A Unified Embedding Space for Ecological Applications

arXiv: https://arxiv.org/abs/2411.00683 · HTML: https://mlsanity.com/paper/2411.00683/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [LiT: Zero-Shot Transfer with Locked-image text Tuning](https://mlsanity.com/paper/2111.07991/) (2021)
- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [Patching open-vocabulary models by interpolating weights](https://mlsanity.com/paper/2208.05592/) (2022)
- [GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization](https://mlsanity.com/paper/2309.16020/) (2023)
- [Learning Deep Representations of Fine-grained Visual Descriptions](https://mlsanity.com/paper/1605.05395/) (2016)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery](https://mlsanity.com/paper/2311.17179/) (2023)
- [Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model](https://mlsanity.com/paper/2208.03987/) (2022)
- [Presence-Only Geographical Priors for Fine-Grained Image Classification](https://mlsanity.com/paper/1906.05272/) (2019)
- [Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2403.05419/) (2024)
- [Factorized Contrastive Learning: Going Beyond Multi-view Redundancy](https://mlsanity.com/paper/2306.05268/) (2023)
- [Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment](https://mlsanity.com/paper/2312.06960/) (2023)
- [Spatial Implicit Neural Representations for Global-Scale Species Mapping](https://mlsanity.com/paper/2306.02564/) (2023)
- [OmniVec: Learning robust representations with cross modal sharing](https://mlsanity.com/paper/2311.05709/) (2023)
- [MetaFormer: A Unified Meta Framework for Fine-Grained Recognition](https://mlsanity.com/paper/2203.02751/) (2022)
- [Sat2Cap: Mapping Fine-Grained Textual Descriptions from Satellite Images](https://mlsanity.com/paper/2307.15904/) (2023)
- [GeoPlant: Spatial Plant Species Prediction Dataset](https://mlsanity.com/paper/2408.13928/) (2024)
- [Learning Tri-modal Embeddings for Zero-Shot Soundscape Mapping](https://mlsanity.com/paper/2309.10667/) (2023)
- [BirdSAT: Cross-View Contrastive Masked Autoencoders for Bird Species Classification and Mapping](https://mlsanity.com/paper/2310.19168/) (2023)
- [Contrastive ground-level image and remote sensing pre-training improves representation learning for natural world imagery](https://mlsanity.com/paper/2409.19439/) (2024)
- [GEOBIND: Binding Text, Image, and Audio through Satellite Images](https://mlsanity.com/paper/2404.11720/) (2024)
- [LD-SDM: Language-Driven Hierarchical Species Distribution Modeling](https://mlsanity.com/paper/2312.08334/) (2023)

## Built on by (most cited first, 30 of 35)

- [AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities](https://mlsanity.com/paper/2412.14123/) (2024)
- [BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning](https://mlsanity.com/paper/2505.23883/) (2025)
- [OmniEarth-Bench: Towards Holistic Evaluation of Earth's Six Spheres and Cross-Spheres Interactions with Multimodal Observational Earth Data](https://mlsanity.com/paper/2505.23522/) (2025)
- [WildSAT: Learning Satellite Image Representations from Wildlife Observations](https://mlsanity.com/paper/2412.14428/) (2024)
- [GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations](https://mlsanity.com/paper/2503.16683/) (2025)
- [Measuring the Intrinsic Dimension of Earth Representations](https://mlsanity.com/paper/2511.02101/) (2025)
- [MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training](https://mlsanity.com/paper/2602.06285/) (2026)
- [Feedforward Few-shot Species Range Estimation](https://mlsanity.com/paper/2502.14977/) (2025)
- [Mind the (Data) Gap: Evaluating Vision Systems in Small Data Applications](https://mlsanity.com/paper/2504.06486/) (2025)
- [EcoWikiRS: Learning Ecological Representation of Satellite Images from Weak Supervision with Species Observations and Wikipedia](https://mlsanity.com/paper/2504.19742/) (2025)
- [Global and Local Entailment Learning for Natural World Imagery](https://mlsanity.com/paper/2506.21476/) (2025)
- [BioCAP: Exploiting Synthetic Captions Beyond Labels in Biological Foundation Models](https://mlsanity.com/paper/2510.20095/) (2025)
- [TaxaAdapter: Vision Taxonomy Models are Key to Fine-grained Image Generation over the Tree of Life](https://mlsanity.com/paper/2603.26128/) (2026)
- [UNIGEOCLIP: Unified Geospatial Contrastive Learning](https://mlsanity.com/paper/2604.11668/) (2026)
- [Tessellating The Earth](https://mlsanity.com/paper/2606.27514/) (2026)
- [Mapping biodiversity at very-high resolution in Europe](https://mlsanity.com/paper/2504.05231/) (2025)
- [TimeSenCLIP: A Time Series Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2508.11919/) (2025)
- [ProM3E: Probabilistic Masked MultiModal Embedding Model for Ecology](https://mlsanity.com/paper/2511.02946/) (2025)
- [Interpretable Perception and Reasoning for Audiovisual Geolocation](https://mlsanity.com/paper/2603.05708/) (2026)
- [Sat2Sound: A Unified Framework for Zero-Shot Soundscape Mapping](https://mlsanity.com/paper/2505.13777/) (2025)
- [CrypticBio: A Large Multimodal Dataset for Visually Confusing Biodiversity](https://mlsanity.com/paper/2505.14707/) (2025)
- [Unsupervised Urban Tree Biodiversity Mapping from Street-Level Imagery Using Spatially-Aware Visual Clustering](https://mlsanity.com/paper/2508.13814/) (2025)
- [Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors](https://mlsanity.com/paper/2512.15748/) (2025)
- [MMLANDMARKS: a Cross-View Instance-Level Benchmark for Geo-Spatial Understanding](https://mlsanity.com/paper/2512.17492/) (2025)
- [Compact Hypercube Embeddings for Fast Text-based Wildlife Observation Retrieval](https://mlsanity.com/paper/2601.22783/) (2026)
- [Audio-to-Image Bird Species Retrieval without Audio-Image Pairs via Text Distillation](https://mlsanity.com/paper/2602.00681/) (2026)
- [TerraDiT: Point-Conditioned Diffusion Transformer for Satellite Image Synthesis](https://mlsanity.com/paper/2603.02172/) (2026)
- [BioVITA: Biological Dataset, Model, and Benchmark for Visual-Textual-Acoustic Alignment](https://mlsanity.com/paper/2603.23883/) (2026)
- [TrajGANR: Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations](https://mlsanity.com/paper/2605.06990/) (2026)
- [Multi-Modal Contrastive Learning for Implicit Earth Embeddings via Location Tying](https://mlsanity.com/paper/2606.20167/) (2026)

## Abstract (arXiv)

We present TaxaBind, a unified embedding space for characterizing any species of interest. TaxaBind is a multimodal embedding space across six modalities: ground-level images of species, geographic location, satellite image, text, audio, and environmental features, useful for solving ecological problems. To learn this joint embedding space, we leverage ground-level images of species as a binding modality. We propose multimodal patching, a technique for effectively distilling the knowledge from various modalities into the binding modality. We construct two large datasets for pretraining: iSatNat with species images and satellite images, and iSoundNat with species images and audio. Additionally, we introduce TaxaBench-8k, a diverse multimodal dataset with six paired modalities for evaluating deep learning models on ecological tasks. Experiments with TaxaBind demonstrate its strong zero-shot and emergent capabilities on a range of tasks including species classification, cross-model retrieval, and audio classification. The datasets and models are made available at https://github.com/mvrl/TaxaBind.