# MixBCT: Towards Self-Adapting Backward-Compatible Training

arXiv: https://arxiv.org/abs/2308.06948 · HTML: https://mlsanity.com/paper/2308.06948/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 52 references that are in the map.
- First posted on arXiv 14 Aug 2023.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [LLM-Pruner: On the Structural Pruning of Large Language Models](https://mlsanity.com/paper/2305.11627/) (2023)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022)
- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [Correlation Congruence for Knowledge Distillation](https://mlsanity.com/paper/1904.01802/) (2019)
- [Post-training Quantization on Diffusion Models](https://mlsanity.com/paper/2211.15736/) (2022)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Cross-Layer Distillation with Semantic Calibration](https://mlsanity.com/paper/2012.03236/) (2020)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Towards Backward-Compatible Representation Learning](https://mlsanity.com/paper/2003.11942/) (2020)
- [Fast Human Pose Estimation](https://mlsanity.com/paper/1811.05419/) (2018)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [Improved Residual Networks for Image and Video Recognition](https://mlsanity.com/paper/2004.04989/) (2020)
- [Learning Student Networks via Feature Embedding](https://mlsanity.com/paper/1812.06597/) (2018)
- [Learning Compatible Embeddings](https://mlsanity.com/paper/2108.01958/) (2021)
- [Conditional Teacher-Student Learning](https://mlsanity.com/paper/1904.12399/) (2019)
- [Unified Representation Learning for Cross Model Compatibility](https://mlsanity.com/paper/2008.04821/) (2020)
- [Towards Universal Backward-Compatible Representation Learning](https://mlsanity.com/paper/2203.01583/) (2022)
- [Domain Re-Modulation for Few-Shot Generative Domain Adaptation](https://mlsanity.com/paper/2302.02550/) (2023)
- [Learning Backward Compatible Embeddings](https://mlsanity.com/paper/2206.03040/) (2022)

## Built on by (most cited first, 2 of 2)

- [Prototype Perturbation for Relaxing Alignment Constraints in Backward-Compatible Learning](https://mlsanity.com/paper/2503.14824/) (2025)
- [Bi-C2R: Bidirectional Continual Compatible Representation for Re-indexing Free Lifelong Person Re-identification](https://mlsanity.com/paper/2512.25000/) (2025)

## Abstract (arXiv)

Backward-compatible training circumvents the need for expensive updates to the old gallery database when deploying an advanced new model in the retrieval system. Previous methods achieved backward compatibility by aligning prototypes of the new model with the old one, yet they often overlooked the distribution of old features, limiting their effectiveness when the low quality of the old model results in a weakly feature discriminability. Instance-based methods like L2 regression take into account the distribution of old features but impose strong constraints on the performance of the new model itself. In this paper, we propose MixBCT, a simple yet highly effective backward-compatible training method that serves as a unified framework for old models of varying qualities. We construct a single loss function applied to mixed old and new features to facilitate backward-compatible training, which adaptively adjusts the constraint domain for new features based on the distribution of old features. We conducted extensive experiments on the large-scale face recognition datasets MS1Mv3 and IJB-C to verify the effectiveness of our method. The experimental results clearly demonstrate its superiority over previous methods. Code is available at https://github.com/yuleung/MixBCT .