# CoReS: Compatible Representations via Stationarity arXiv: https://arxiv.org/abs/2111.07632 · HTML: https://mlsanity.com/paper/2111.07632/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Builds on 48 of its 87 references that are in the map. - First posted on arXiv 15 Nov 2021. ## Builds on (most cited first, 30 of 48) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020) - [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017) - [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015) - [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017) - [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016) - [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017) - [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014) - [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020) - [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017) - [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020) - [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019) - [Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training](https://mlsanity.com/paper/1712.01887/) (2017) - [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014) - [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020) - [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021) - [What is being transferred in transfer learning?](https://mlsanity.com/paper/2008.11687/) (2020) - [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014) - [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020) - [Large-Margin Softmax Loss for Convolutional Neural Networks](https://mlsanity.com/paper/1612.02295/) (2016) - [Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines](https://mlsanity.com/paper/1810.12488/) (2018) - [A Geometric Analysis of Neural Collapse with Unconstrained Features](https://mlsanity.com/paper/2105.02375/) (2021) - [Uniform convergence may be unable to explain generalization in deep learning](https://mlsanity.com/paper/1902.04742/) (2019) - [Convergent Learning: Do different neural networks learn the same representations?](https://mlsanity.com/paper/1511.07543/) (2015) - [Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training](https://mlsanity.com/paper/2101.12699/) (2021) - [Deep Learning Face Representation by Joint Identification-Verification](https://mlsanity.com/paper/1406.4773/) (2014) - [Particular object retrieval with integral max-pooling of CNN activations](https://mlsanity.com/paper/1511.05879/) (2015) - [Person Re-identification in the Wild](https://mlsanity.com/paper/1604.02531/) (2016) - [Deep Image Retrieval: Learning global representations for image search](https://mlsanity.com/paper/1604.01325/) (2016) ## Built on by (most cited first, 14 of 14) - [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022) - [Latent Space Translation via Semantic Alignment](https://mlsanity.com/paper/2311.00664/) (2023) - [Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants](https://mlsanity.com/paper/2308.01746/) (2023) - [Latent Space Translation via Inverse Relative Projection](https://mlsanity.com/paper/2406.15057/) (2024) - [Bootstrapping Parallel Anchors for Relative Representations](https://mlsanity.com/paper/2303.00721/) (2023) - [boldsymbolλ-Orthogonality Regularization for Compatible Representation Learning](https://mlsanity.com/paper/2509.16664/) (2025) - [Global Minimizers of Sigmoid Contrastive Loss](https://mlsanity.com/paper/2509.18552/) (2025) - [CL2R: Compatible Lifelong Learning Representations](https://mlsanity.com/paper/2211.09032/) (2022) - [Contrastive Supervised Distillation for Continual Representation Learning](https://mlsanity.com/paper/2205.05476/) (2022) - [Backward-Compatible Aligned Representations via an Orthogonal Transformation Layer](https://mlsanity.com/paper/2408.08793/) (2024) - [Prototype Perturbation for Relaxing Alignment Constraints in Backward-Compatible Learning](https://mlsanity.com/paper/2503.14824/) (2025) - [A Stationary (and Therefore Compatible) Representation is All You Need](https://mlsanity.com/paper/2606.12488/) (2026) - [A Stitch in Time Saves Nine: Preserving Policy Compatibility Under Perception Updates in End-to-End Autonomous Driving](https://mlsanity.com/paper/2606.21509/) (2026) - [Spherical Interpolation for Backward-Compatible Multimodal Representations](https://mlsanity.com/paper/2609.39836/) (2026) ## Abstract (arXiv) Compatible features enable the direct comparison of old and new learned features allowing to use them interchangeably over time. In visual search systems, this eliminates the need to extract new features from the gallery-set when the representation model is upgraded with novel data. This has a big value in real applications as re-indexing the gallery-set can be computationally expensive when the gallery-set is large, or even infeasible due to privacy or other concerns of the application. In this paper, we propose CoReS, a new training procedure to learn representations that are textitcompatible with those previously learned, grounding on the stationarity of the features as provided by fixed classifiers based on polytopes. With this solution, classes are maximally separated in the representation space and maintain their spatial configuration stationary as new classes are added, so that there is no need to learn any mappings between representations nor to impose pairwise training with the previously learned model. We demonstrate that our training procedure largely outperforms the current state of the art and is particularly effective in the case of multiple upgrades of the training-set, which is the typical case in real applications.