# CouCE: A Unified Causal Framework for Debiased Deep Metric Learning

arXiv: https://arxiv.org/abs/2606.30365 · HTML: https://mlsanity.com/paper/2606.30365/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 23 of 23)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [A Fourier-based Framework for Domain Generalization](https://mlsanity.com/paper/2105.11120/) (2021)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [Representation Learning via Invariant Causal Mechanisms](https://mlsanity.com/paper/2010.07922/) (2020)
- [Proxy Anchor Loss for Deep Metric Learning](https://mlsanity.com/paper/2003.13911/) (2020)
- [SoftTriple Loss: Deep Metric Learning Without Triplet Sampling](https://mlsanity.com/paper/1909.05235/) (2019)
- [HIER: Metric Learning Beyond Class Labels via Hierarchical Regularization](https://mlsanity.com/paper/2212.14258/) (2022)
- [Learning Causally Disentangled Representations via the Principle of Independent Causal Mechanisms](https://mlsanity.com/paper/2306.01213/) (2023)
- [Semantic Data Augmentation based Distance Metric Learning for Domain Generalization](https://mlsanity.com/paper/2208.02803/) (2022)
- [Towards Improved Proxy-based Deep Metric Learning via Data-Augmented Domain Adaptation](https://mlsanity.com/paper/2401.00617/) (2024)
- [CausalRec: Causal Inference for Visual Debiasing in Visually-Aware Recommendation](https://mlsanity.com/paper/2107.02390/) (2021)
- [Towards Combating Frequency Simplicity-biased Learning for Domain Generalization](https://mlsanity.com/paper/2410.16146/) (2024)
- [Potential Field Based Deep Metric Learning](https://mlsanity.com/paper/2405.18560/) (2024)
- [BackMix: Mitigating Shortcut Learning in Echocardiography with Minimal Supervision](https://mlsanity.com/paper/2406.19148/) (2024)
- [PD-Loss: Proxy-Decidability for Efficient Metric Learning](https://mlsanity.com/paper/2508.17082/) (2025)

## Abstract (arXiv)

Deep Metric Learning (DML) often struggles with zero-shot generalization because standard objectives inherently capture what co-occurs rather than what causes similarity. Consequently, DML models are vulnerable to shortcut learning driven by two structurally distinct confounders: background spurious correlations (which create backdoor paths via scene context) and foreground nuisance perturbations (which inject non-semantic variations like pose or illumination). Although existing methods have proposed targeted solutions for each pathway individually, none can simultaneously address both due to their fundamentally distinct causal roles. To bridge this gap, we propose the Counterfactual Causal Embedding (CouCE), a unified causal framework that explicitly models and neutralizes both confounders. Specifically, we introduce Orthogonal Dictionary-Based Backdoor Adjustment (ODBA), which isolates spurious background patterns into a variance-gated dictionary and stably disentangles them from the learned embeddings via soft orthogonal regularization. Simultaneously, we propose Multi-Scale Randomized Causal Intervention (MSRCI) to enforce causal invariance against foreground nuisances through multi-scale Fourier amplitude randomization and a symmetric KL invariance constraint. Notably, CouCE seamlessly integrates with any proxy-based loss, incurring modest training overhead without requiring architectural modifications during inference. Extensive experiments on CUB-200-2011, Cars-196, and Stanford Online Products demonstrate that CouCE consistently achieves state-of-the-art performance, providing a principled and robust solution for debiased DML.