mlsanityData to 14 Oct 2026

Learning Compatible Embeddings

arXiv 2108.01958 ↗First posted 4 Aug 2021cs.CVcited by 37

Not on the map. Why?

Abstract (arXiv)

Achieving backward compatibility when rolling out new models can highly reduce costs or even bypass feature re-encoding of existing gallery images for in-production visual retrieval systems. Previous related works usually leverage losses used in knowledge distillation which can cause performance degradations or not guarantee compatibility. To address these issues, we propose a general framework called Learning Compatible Embeddings (LCE) which is applicable for both cross model compatibility and compatible training in direct/forward/backward manners. Our compatibility is achieved by aligning class centers between models directly or via a transformation, and restricting more compact intra-class distributions for the new model. Experiments are conducted in extensive scenarios such as changes of training dataset, loss functions, network architectures as well as feature dimensions, and demonstrate that LCE efficiently enables model compatibility with marginal sacrifices of accuracies. The code will be available at https://github.com/IrvingMeng/LCE.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 4 Aug 2021.

Builds on

39 of 59 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  4. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  5. 2020Knowledge Distillation: A Surveycited by 1,603
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  7. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  8. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  9. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 971
  10. 2019Relational Knowledge Distillationcited by 889
  11. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 786
  12. 2016MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognitioncited by 624
27 more
  1. 2018Rethinking the Value of Network Pruningcited by 599
  2. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  3. 2015Towards Open Set Deep Networkscited by 538
  4. 2013Do Deep Nets Really Need to be Deep?cited by 474
  5. 2017Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)cited by 473
  6. 2021MagFace: A Universal Representation for Face Recognition and Quality Assessmentcited by 334
  7. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 316
  8. 2019Correlation Congruence for Knowledge Distillationcited by 285
  9. 2018Paraphrasing Complex Network: Network Compression via Factor Transfercited by 260
  10. 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
  11. 2018MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devicescited by 182
  12. 2016Deep Image Retrieval: Learning global representations for image searchcited by 147
  13. 2017L2-constrained Softmax Loss for Discriminative Face Verificationcited by 96
  14. 2018Features for Multi-Target Multi-Camera Tracking and Re-Identificationcited by 88
  15. 2020Towards Backward-Compatible Representation Learningcited by 75
  16. 2020Appearance-Preserving 3D Convolution for Video-based Person Re-identificationcited by 63
  17. 2015DeepID3: Face Recognition with Very Deep Neural Networkscited by 62
  18. 2017DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfercited by 56
  19. 2018Learning Student Networks via Feature Embeddingcited by 45
  20. 2017Joint Geometrical and Statistical Alignment for Visual Domain Adaptationcited by 43
  21. 2019A survey of advances in vision-based vehicle re-identificationcited by 36
  22. 2018Balanced Distribution Adaptation for Transfer Learningcited by 32
  23. 2020Unified Representation Learning for Cross Model Compatibilitycited by 30
  24. 2017Rocket Launching: A Universal and Efficient Framework for Training Well-performing Light Netcited by 27
  25. 2020Matching Guided Distillationcited by 18
  26. 2021Searching for Alignment in Face Recognitioncited by 12
  27. 2021PoseFace: Pose-Invariant Features and Pose-Adaptive Loss for Face Recognitioncited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 13; 2023: 11; 2024: 3; 2025: 5; 2026: 3.

37 papers.

Sort:

By area:

  1. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 127
  2. 2022Towards Universal Backward-Compatible Representation Learningcited by 21
  3. 2021Forward Compatible Training for Large-Scale Embedding Retrieval Systemscited by 19
  4. 2022Improving Federated Learning Face Recognition via Privacy-Agnostic Clusterscited by 18
  5. 2022Learning Backward Compatible Embeddingscited by 16
  6. 2021CoReS: Compatible Representations via Stationaritycited by 14
  7. 2023Boundary-aware Backward-Compatible Representation via Adversarial Learning in Image Retrievalcited by 11
  8. 2022Privacy-Preserving Model Upgrades with Bidirectional Compatible Training in Image Retrievalcited by 9
  9. 2022BT2: Backward-compatible Training with Basis Transformationcited by 8
  10. 2023Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Promptcited by 8
  11. 2025boldsymbolλ-Orthogonality Regularization for Compatible Representation Learningcited by 8
  12. 2022Hot-Refresh Model Upgrades with Regression-Alleviating Compatible Training in Image Retrievalcited by 7
25 more
  1. 2022When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework and A New Benchmarkcited by 7
  2. 2023FastFill: Efficient Compatible Model Updatecited by 7
  3. 2022CL2R: Compatible Lifelong Learning Representationscited by 6
  4. 2023Metric Compatible Training for Online Backfilling in Large-Scale Retrievalcited by 6
  5. 2023Improving Prediction Backward-Compatiblility in NLP Model Upgrade with Gated Fusioncited by 6
  6. 2023Binary Embedding-based Retrieval at Tencentcited by 6
  7. 2022Darwinian Model Upgrades: Model Evolving with Selective Compatibilitycited by 5
  8. 2024Towards Cross-modal Backward-compatible Representation Learning for Vision-Language Modelscited by 5
  9. 2025Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learningcited by 4
  10. 2022Dynamic Gradient Reactivation for Backward Compatible Person Re-identificationcited by 3
  11. 2023Learning Repeatable Speech Embeddings Using An Intra-class Correlation Regularizercited by 3
  12. 2024Backward-Compatible Aligned Representations via an Orthogonal Transformation Layercited by 3
  13. 2022Towards Privacy-Preserving, Real-Time and Lossless Feature Matchingcited by 2
  14. 2023Asymmetric Image Retrieval with Cross Model Compatible Ensemblescited by 2
  15. 2023MixBCT: Towards Self-Adapting Backward-Compatible Trainingcited by 2
  16. 2024Lifelong Person Re-Identification with Backward-Compatibilitycited by 2
  17. 2025When Embedding Models Meet: Procrustes Bounds and Applicationscited by 2
  18. 2025Towards Test-time Efficient Visual Place Recognition via Asymmetric Query Processingcited by 2
  19. 2022Basket-based Softmaxcited by 1
  20. 2026A Stationary (and Therefore Compatible) Representation is All You Needcited by 1
  21. 2023RAFEN -- Regularized Alignment Framework for Embeddings of Nodescited by 0
  22. 2023Learning Unified Representations for Multi-Resolution Face Recognitioncited by 0
  23. 2025Learning Compatible Multi-Prize Subnetworks for Asymmetric Retrievalcited by 0
  24. 2026Multi-modal Knowledge Preserving Adapter for Embedding Backward Compatibilitycited by 0
  25. 2026Spherical Interpolation for Backward-Compatible Multimodal Representationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ReinPool: Reinforcement Learning Pooling Multi-Vector Embeddings for Retrieval System
  2. 2025ABC: Achieving Better Control of Multimodal Embeddings using VLMs
  3. 2025MIEB: Massive Image Embedding Benchmark
  4. 2024Structure Similarity Preservation Learning for Asymmetric Image Retrieval
  5. 2023SynCDR : Training Cross Domain Retrieval Models with Synthetic Data
  6. 2024MaskInversion: Localized Embeddings via Optimization of Explainability Maps
  7. 2026SHED: Style-Homogenized Embedding Alignment for Domain Generalization
  8. 2025RzenEmbed: Towards Comprehensive Multimodal Retrieval