# Reviving the Context: Camera Trap Species Classification as Link Prediction on Multimodal Knowledge Graphs

arXiv: https://arxiv.org/abs/2401.00608 · HTML: https://mlsanity.com/paper/2401.00608/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 74 references that are in the map.
- First posted on arXiv 31 Dec 2023.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network](https://mlsanity.com/paper/1712.02121/) (2017)
- [Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks](https://mlsanity.com/paper/1912.09393/) (2019)
- [The iWildCam 2020 Competition Dataset](https://mlsanity.com/paper/2004.10340/) (2020)
- [Sequence-to-Sequence Knowledge Graph Completion and Question Answering](https://mlsanity.com/paper/2203.10321/) (2022)
- [A Re-evaluation of Knowledge Graph Completion Methods](https://mlsanity.com/paper/1911.03903/) (2019)
- [The More You Know: Using Knowledge Graphs for Image Classification](https://mlsanity.com/paper/1612.04844/) (2016)
- [Do Convolutional Neural Networks Learn Class Hierarchy?](https://mlsanity.com/paper/1710.06501/) (2017)
- [HittER: Hierarchical Transformers for Knowledge Graph Embeddings](https://mlsanity.com/paper/2008.12813/) (2020)
- [Geo-Aware Networks for Fine-Grained Recognition](https://mlsanity.com/paper/1906.01737/) (2019)
- [A deep active learning system for species identification and counting in camera trap images](https://mlsanity.com/paper/1910.09716/) (2019)
- [Embedding Multimodal Relational Data for Knowledge Base Completion](https://mlsanity.com/paper/1809.01341/) (2018)
- [IMF: Interactive Multimodal Fusion Model for Link Prediction](https://mlsanity.com/paper/2303.10816/) (2023)
- [KBLRN : End-to-End Learning of Knowledge Base Representations with Latent, Relational, and Numerical Features](https://mlsanity.com/paper/1709.04676/) (2017)
- [Deep Learning Object Detection Methods for Ecological Camera Trap Data](https://mlsanity.com/paper/1803.10842/) (2018)
- [The iWildCam 2018 Challenge Dataset](https://mlsanity.com/paper/1904.05986/) (2019)
- [Automated wildlife image classification: An active learning tool for ecological applications](https://mlsanity.com/paper/2303.15823/) (2023)
- [A Retrieve-and-Read Framework for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2212.09724/) (2022)
- [Love Thy Neighbors: Image Annotation by Exploiting Image Metadata](https://mlsanity.com/paper/1508.07647/) (2015)
- [Ontology-based n-ball Concept Embeddings Informing Few-shot Image Classification](https://mlsanity.com/paper/2109.09063/) (2021)

## Built on by (most cited first, 4 of 4)

- [Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey](https://mlsanity.com/paper/2402.05391/) (2024)
- [AI-Driven Real-Time Monitoring of Ground-Nesting Birds: A Case Study on Curlew Detection Using YOLOv10](https://mlsanity.com/paper/2411.15263/) (2024)
- [Understanding the Embedding Models on Hyper-relational Knowledge Graph](https://mlsanity.com/paper/2508.03280/) (2025)
- [Automatic Image-Level Morphological Trait Annotation for Organismal Images](https://mlsanity.com/paper/2604.01619/) (2026)

## Abstract (arXiv)

Camera traps are important tools in animal ecology for biodiversity monitoring and conservation. However, their practical application is limited by issues such as poor generalization to new and unseen locations. Images are typically associated with diverse forms of context, which may exist in different modalities. In this work, we exploit the structured context linked to camera trap images to boost out-of-distribution generalization for species classification tasks in camera traps. For instance, a picture of a wild animal could be linked to details about the time and place it was captured, as well as structured biological knowledge about the animal species. While often overlooked by existing studies, incorporating such context offers several potential benefits for better image understanding, such as addressing data scarcity and enhancing generalization. However, effectively incorporating such heterogeneous context into the visual domain is a challenging problem. To address this, we propose a novel framework that transforms species classification as link prediction in a multimodal knowledge graph (KG). This framework enables the seamless integration of diverse multimodal contexts for visual recognition. We apply this framework for out-of-distribution species classification on the iWildCam2020-WILDS and Snapshot Mountain Zebra datasets and achieve competitive performance with state-of-the-art approaches. Furthermore, our framework enhances sample efficiency for recognizing under-represented species.