# Cross-Modal Subspace Learning with Scheduled Adaptive Margin Constraints

arXiv: https://arxiv.org/abs/1909.13733 · HTML: https://mlsanity.com/paper/1909.13733/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2019.

## Built on by (most cited first, 4 of 4)

- [Relevance-based Margin for Contrastively-trained Video Retrieval Models](https://mlsanity.com/paper/2204.13001/) (2022)
- [Improving Video Retrieval by Adaptive Margin](https://mlsanity.com/paper/2303.05093/) (2023)
- [Leveraging Generative Language Models for Weakly Supervised Sentence Component Analysis in Video-Language Joint Learning](https://mlsanity.com/paper/2312.06699/) (2023)
- [Maintaining Difficulty: A Margin Scheduler for Triplet Loss in Siamese Networks Training](https://mlsanity.com/paper/2603.26389/) (2026)

## Abstract (arXiv)

Cross-modal embeddings, between textual and visual modalities, aim to organise multimodal instances by their semantic correlations. State-of-the-art approaches use maximum-margin methods, based on the hinge-loss, to enforce a constant margin m, to separate projections of multimodal instances from different categories. In this paper, we propose a novel scheduled adaptive maximum-margin (SAM) formulation that infers triplet-specific constraints during training, therefore organising instances by adaptively enforcing inter-category and inter-modality correlations. This is supported by a scheduled adaptive margin function, that is smoothly activated, replacing a static margin by an adaptively inferred one reflecting triplet-specific semantic correlations while accounting for the incremental learning behaviour of neural networks to enforce category cluster formation and enforcement. Experiments on widely used datasets show that our model improved upon state-of-the-art approaches, by achieving a relative improvement of up to ~12.5% over the second best method, thus confirming the effectiveness of our scheduled adaptive margin formulation.