# Face, Whole-Person, and Object Classification in a Unified Space Via The Interleaved Multi-Domain Identity Curriculum

arXiv: https://arxiv.org/abs/2511.19846 · HTML: https://mlsanity.com/paper/2511.19846/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Builds on (most cited first, 19 of 19)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Person Transfer GAN to Bridge Domain Gap for Person Re-Identification](https://mlsanity.com/paper/1711.08565/) (2017)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [EVA-02: A Visual Representation for Neon Genesis](https://mlsanity.com/paper/2303.11331/) (2023)
- [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021)
- [Multimodal Autoregressive Pre-training of Large Vision Encoders](https://mlsanity.com/paper/2411.14402/) (2024)
- [UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles](https://mlsanity.com/paper/2104.00946/) (2021)
- [Person Re-identification by Contour Sketch under Moderate Clothing Change](https://mlsanity.com/paper/2002.02295/) (2020)
- [Don't Stop Learning: Towards Continual Learning for the CLIP Model](https://mlsanity.com/paper/2207.09248/) (2022)
- [Expanding Accurate Person Recognition to New Altitudes and Ranges: The BRIAR Dataset](https://mlsanity.com/paper/2211.01917/) (2022)
- [FRoundation: Are Foundation Models Ready for Face Recognition?](https://mlsanity.com/paper/2410.23831/) (2024)
- [Parameter Efficient Fine-tuning of Self-supervised ViTs without Catastrophic Forgetting](https://mlsanity.com/paper/2404.17245/) (2024)
- [PETALface: Parameter Efficient Transfer Learning for Low-resolution Face Recognition](https://mlsanity.com/paper/2412.07771/) (2024)
- [Dissecting Human Body Representations in Deep Networks Trained for Person Identification](https://mlsanity.com/paper/2502.15934/) (2025)
- [Interleaved Multitask Learning with Energy Modulated Learning Progress](https://mlsanity.com/paper/2504.00707/) (2025)
- [CLIP Unreasonable Potential in Single-Shot Face Recognition](https://mlsanity.com/paper/2411.12319/) (2024)
- [Not All Starting Points Are Equal: Pre-trained Priors and Their Outsized Impact on Person Identification](https://mlsanity.com/paper/2507.17640/) (2025)

## Abstract (arXiv)

Vision foundation models can perform generalized object classification in zero-shot mode, and face/person recognition when they are fine-tuned. However, fine-tuned models suffer from catastrophic forgetting. We create models that perform four tasks (object recognition, face recognition from high- and low-quality images, and person recognition from whole-body images) in a single embedding space -- without incurring substantial catastrophic forgetting. To accomplish this, we introduce two variants of the Interleaved Multi-Domain Identity Curriculum (IMIC): a gradient-coupled, interleaving training schedule that fine-tunes a foundation backbone simultaneously on all four tasks. The IMIC method proved effective with three foundation model bases: DINOv3, CLIP, and EVA-02. Two of these (EVA-02 and CLIP) performed comparably with domain experts on all four tasks concurrently and were more accurate than humans at multi-tasking across face, body, and object datasets. Further, we demonstrate that our approach does not substantially harm out-of-distribution generalization, thus maintaining a key property of foundation models. Analysis of the most accurate model variants (EVA-02 + IMIC A and B) showed linearly separable representations of the four tasks in the unified embedding space, but with substantial sharing of features across tasks. Fewer than 100 PCs calculated from any one task could perform all other tasks with nearly zero performance degradation.