mlsanityData to 7 Oct 2026

Hierarchy-Guided Multimodal Representation Learning for Taxonomic Inference

arXiv 2603.25573 ↗First posted 26 Mar 2026cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Accurate biodiversity identification from large-scale field data is a foundational problem with direct impact on ecology, conservation, and environmental monitoring. In practice, the core task is taxonomic prediction - inferring order, family, genus, or species from imperfect inputs such as specimen images, DNA barcodes, or both. Existing multimodal methods often treat taxonomy as a flat label space and therefore fail to encode the hierarchical structure of biological classification, which is critical for robustness under noise and missing modalities. We present two end-to-end variants for hierarchy-aware multimodal learning: CLiBD-HiR, which introduces Hierarchical Information Regularization (HiR) to shape embedding geometry across taxonomic levels, yielding structured and noise-robust representations; and CLiBD-HiR-Fuse, which additionally trains a lightweight fusion predictor that supports image-only, DNA-only, or joint inference and is resilient to modality corruption. Across large-scale biodiversity benchmarks, our approach improves taxonomic classification accuracy by over 14 percent compared to strong multimodal baselines, with particularly large gains under partial and corrupted DNA conditions. These results highlight that explicitly encoding biological hierarchy, together with flexible fusion, is key for practical biodiversity foundation models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 17 references that are in the map. First posted on arXiv 26 Mar 2026.

Builds on

10 of 17 references.

References · 10

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Supervised Contrastive Learningcited by 2,803
  3. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  4. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  5. 2023BioCLIP: A Vision Foundation Model for the Tree of Lifecited by 159
  6. 2024FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusioncited by 67
  7. 2022Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Frameworkcited by 59
  8. 2023Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusioncited by 54
  9. 2023A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Datasetcited by 30
  10. 2024CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scalecited by 18

Built on by

Not cited yet.