# From Visual to Multimodal: Systematic Ablation of Encoders and Fusion Strategies in Animal Identification

arXiv: https://arxiv.org/abs/2603.02270 · HTML: https://mlsanity.com/paper/2603.02270/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 73 references that are in the map.
- First posted on arXiv 28 Feb 2026.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [YOLOv12: Attention-Centric Real-Time Object Detectors](https://mlsanity.com/paper/2502.12524/) (2025)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [WildlifeDatasets: An open-source toolkit for animal re-identification](https://mlsanity.com/paper/2311.09118/) (2023)
- [Perspectives on individual animal identification from biology and computer vision](https://mlsanity.com/paper/2103.00560/) (2021)
- [AnimalWeb: A Large-Scale Hierarchical Dataset of Annotated Animal Faces](https://mlsanity.com/paper/1909.04951/) (2019)
- [SealID: Saimaa ringed seal re-identification dataset](https://mlsanity.com/paper/2206.02260/) (2022)
- [WildlifeReID-10k: Wildlife re-identification dataset with 10k individual animals](https://mlsanity.com/paper/2406.09211/) (2024)
- [Similarity Learning Networks for Animal Individual Re-Identification -- Beyond the Capabilities of a Human Observer](https://mlsanity.com/paper/1902.09324/) (2019)
- [Multispecies Animal Re-ID Using a Large Community-Curated Dataset](https://mlsanity.com/paper/2412.05602/) (2024)
- [PetFace: A Large-Scale Dataset and Benchmark for Animal Identification](https://mlsanity.com/paper/2407.13555/) (2024)
- [WildFusion: Individual Animal Identification with Calibrated Similarity Fusion](https://mlsanity.com/paper/2408.12934/) (2024)
- [MetaWild: A Multimodal Dataset for Animal Re-Identification with Environmental Metadata](https://mlsanity.com/paper/2501.13368/) (2025)
- [OpenAnimals: Revisiting Person Re-Identification for Animals Towards Better Generalization](https://mlsanity.com/paper/2410.00204/) (2024)
- [CATALOG: A Camera Trap Language-guided Contrastive Learning Model](https://mlsanity.com/paper/2412.10624/) (2024)
- [Generalizable Object Re-Identification via Visual In-Context Prompting](https://mlsanity.com/paper/2508.21222/) (2025)
- [Learning Item Representations Directly from Multimodal Features for Effective Recommendation](https://mlsanity.com/paper/2505.04960/) (2025)

## Abstract (arXiv)

Automated animal identification is a practical task for reuniting lost pets with their owners, yet current systems often struggle due to limited dataset scale and reliance on unimodal visual cues. This study introduces a multimodal verification framework that enhances visual features with semantic identity priors derived from synthetic textual descriptions. We constructed a massive training corpus of 1.9 million photographs covering 695,091~unique animals to support this investigation. Through systematic ablation studies, we identified SigLIP2-Giant and E5-Small-v2 as the optimal vision and text backbones. We further evaluated fusion strategies ranging from simple concatenation to adaptive gating to determine the best method for integrating these modalities. Our proposed approach utilizes a gated fusion mechanism and achieved a Top-1 accuracy of 84.28% and an Equal Error Rate of 0.0422 on a comprehensive test protocol. These results represent an 11% improvement over leading unimodal baselines and demonstrate that integrating synthesized semantic descriptions significantly refines decision boundaries in large-scale pet re-identification. Conference update. This work was accepted to the FGVC13 Workshop at CVPR 2026. We thank Vadim Ezhov and Nikolai Shcheglov for their help with the poster presentation and demonstration at the conference.