Deep Metric Learning via Lifted Structured Feature Embedding
Not on the map. Why?
Abstract (arXiv)
Learning the distance metric between pairs of examples is of great importance for learning and visual recognition. With the remarkable success from the state of the art convolutional neural networks, recent works have shown promising results on discriminatively training the networks to learn semantic feature embeddings where similar examples are mapped close to each other and dissimilar examples are mapped farther apart. In this paper, we describe an algorithm for taking full advantage of the training batches in the neural network training by lifting the vector of pairwise distances within the batch to the matrix of pairwise distances. This step enables the algorithm to learn the state of the art feature embedding by optimizing a novel structured prediction objective on the lifted problem. Additionally, we collected Online Products dataset: 120k images of 23k classes of online products for metric learning. Our experiments on the CUB-200-2011, CARS196, and Online Products datasets demonstrate significant improvement over existing deep feature embedding methods on all experimented embedding sizes with the GoogLeNet network.
Key facts. Cited by 449 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Nov 2015.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 114; 2022: 123; 2023: 82; 2024: 66; 2025: 38; 2026: 26.
449 papers.
Sort:
By area:
- 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
- 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
- 2023A Cookbook of Self-Supervised Learningcited by 250
- 2021NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wildcited by 136
- 2023Hyperbolic Image-Text Representationscited by 136
- 2021Fine-Grained Image Analysis with Deep Learning: A Surveycited by 135
- 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
- 2022CyCLIP: Cyclic Contrastive Language-Image Pretrainingcited by 127
- 2022What to Hide from Your Students: Attention-Guided Masked Image Modelingcited by 118
- 2022Hyperbolic Vision Transformers: Combining Improvements in Metric Learningcited by 111
- 2021ResLT: Residual Learning for Long-tailed Recognitioncited by 96
- 2021Training Vision Transformers for Image Retrievalcited by 90
38 more
- 2023Image Captioners Are Scalable Vision Learners Toocited by 89
- 2021ACE: Ally Complementary Experts for Solving Long-Tailed Recognition in One-Shotcited by 86
- 2022A Survey on Long-Tailed Visual Recognitioncited by 75
- 2021Cross Modal Retrieval with Querybank Normalisationcited by 72
- 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
- 2022SUES-200: A Multi-height Multi-scene Cross-view Image Benchmark Across Drone and Satellitecited by 64
- 2021Instance-level Image Retrieval using Reranking Transformerscited by 63
- 2021RSG: A Simple but Effective Module for Learning Imbalanced Datasetscited by 61
- 2022Visual Recognition with Deep Nearest Centroidscited by 59
- 2021Universal Representation Learning from Multiple Domains for Few-shot Classificationcited by 56
- 2021Recall@k Surrogate Loss with Large Batches and Similarity Mixupcited by 49
- 2021Incremental False Negative Detection for Contrastive Learningcited by 48
- 2022Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularizationcited by 45
- 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversioncited by 43
- 2021von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learningcited by 42
- 2021Mutual Contrastive Learning for Visual Representation Learningcited by 42
- 2021SphereFace2: Binary Classification is All You Need for Deep Face Recognitioncited by 42
- 2022Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learningcited by 38
- 2021Boosting the Speed of Entity Alignment 10*: Dual Attention Matching Network with Normalized Hard Sample Miningcited by 37
- 2022ECCV Caption: Correcting False Negatives by Collecting Machine-and-Human-verified Image-Caption Associations for MS-COCOcited by 37
- 2024TIPS: Text-Image Pretraining with Spatial awarenesscited by 37
- 2022Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognitioncited by 36
- 2023Unicom: Universal and Compact Representation Learning for Image Retrievalcited by 36
- 2023Contrastive Difference Predictive Codingcited by 36
- 2021Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learningcited by 35
- 2022Understanding Deep Contrastive Learning via Coordinate-wise Optimizationcited by 35
- 2021Learning Intra-Batch Connections for Deep Metric Learningcited by 34
- 2023Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentationcited by 34
- 2022Non-isotropy Regularization for Proxy-based Deep Metric Learningcited by 33
- 2021GistNet: a Geometric Structure Transfer Network for Long-Tailed Recognitioncited by 31
- 2021Pairwise Supervised Contrastive Learning of Sentence Representationscited by 31
- 2022A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainabilitycited by 31
- 2021Towards Interpretable Deep Metric Learning with Structural Matchingcited by 30
- 2022Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learningcited by 30
- 2024AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Modelscited by 30
- 2024Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Latercited by 30
- 2025ILIAS: Instance-Level Image retrieval At Scalecited by 30
- 2023Towards Universal Image Embeddings: A Large-Scale Dataset and Challenge for Generic Image Representationscited by 28
50 most cited of 449 All 449 (JSON)
- 2026Masked Swingers: Harnessing Data Augmentation to Advance Autoencoders for Self-Supervised Learning29 Sep 2026
- 2026ProtoSeam: Lifting Classifier Training with Latent Gaussian Mixture Models28 Sep 2026
- 2026Multimodal Floorplan Encoding: Learning Dense Modality-Invariant Representations11 Sep 2026
- 2026RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learning2 Sep 2026
- 2026EviRank: Structured Relevance Evidence for Multimodal Image Re-ranking21 Aug 2026
- 2026Illuminating Visual Identity in Universal Multimodal Embeddings3 Aug 2026
- 2026Face and Voice Cross-modal Association with Learning Convex Feature Embedding30 Jul 2026
- 2026Understanding Submodular Information Measure Based Objectives for Representation Learning: A Variance and Separation Perspective30 Jul 2026
- 2026COAST: Context-Aware Differential Learning for Gene Expression Prediction in Spatial Transcriptomics10 Jul 2026
- 2026CouCE: A Unified Causal Framework for Debiased Deep Metric Learning29 Jun 2026
- 2026Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings13 Jun 2026
- 2026Learning to Align Generative Appearance Priors for Fine-grained Image Retrieval11 May 2026
38 more
- 2026TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment13 Apr 2026
- 2026ID-Sim: An Identity-Focused Similarity Metric6 Apr 2026
- 2026ELViS: Efficient Visual Similarity from Local Descriptors that Generalizes Across Domains30 Mar 2026
- 2026PixelSmile: Toward Fine-Grained Facial Expression Editing26 Mar 2026
- 2026Visual Product Search Benchmark17 Mar 2026
- 2026SSR: A Generic Framework for Text-Aided Map Compression for Localization4 Mar 2026
- 2026PointAlign: Feature-Level Alignment Regularization for 3D Vision-Language Models28 Feb 2026
- 2026A Mixed Diet Makes DINO An Omnivorous Vision Encoder27 Feb 2026
- 2026SHaSaM: Submodular Hard Sample Mining for Fair Facial Attribute Recognition5 Feb 2026
- 2026Enhancing Imbalanced Node Classification via Curriculum-Guided Feature Learning and Three-Stage Attention Network3 Feb 2026
- 2026LookBench: A Live and Holistic Open Benchmark for Fashion Image Retrieval21 Jan 2026
- 2026IIR-VLM: In-Context Instance-level Recognition for Large Vision-Language Models20 Jan 2026
- 2026Beyond Seen Bounds: Class-Centric Polarization for Single-Domain Generalized Deep Metric Learning14 Jan 2026
- 2026Dynamic Graph Structure Learning via Resistance Curvature Flow13 Jan 2026
- 2025Training-free Conditional Image Embedding Framework Leveraging Large Vision Language Models26 Dec 2025
- 2025Finer-Personalization Rank: Fine-Grained Retrieval Examines Identity Preservation for Personalized Generation22 Dec 2025
- 2025Language-driven Fine-grained Retrieval6 Dec 2025
- 2025Intelligent Image Search Algorithms Fusing Visual Large Models25 Nov 2025
- 2025A Hybrid Multimodal Deep Learning Framework for Intelligent Fashion Recommendation10 Nov 2025
- 2025DORAEMON: A Unified Library for Visual Object Modeling and Representation Learning at Scale6 Nov 2025
- 2025Instance-Level Composed Image Retrieval29 Oct 2025
- 2025Understanding Self-supervised Contrastive Learning through Supervised Objectives12 Oct 2025
- 2025Instance-Level Generation for Representation Learning10 Oct 2025
- 2025Combined Hyperbolic and Euclidean Soft Triple Loss Beyond the Single Space Deep Metric Learning7 Oct 2025
- 2025Unsupervised Transformer Pre-Training for Images: Self-Distillation, Mean Teachers, and Random Crops4 Oct 2025
- 2025Global-to-Local or Local-to-Global? Enhancing Image Retrieval with Efficient Local Search and Effective Global Re-ranking4 Sep 2025
- 2025Generalizable Object Re-Identification via Visual In-Context Prompting28 Aug 2025
- 2025Infusing fine-grained visual knowledge to Vision-Language Models16 Aug 2025
- 2025GEDAN: Learning the Edit Costs for Graph Edit Distance5 Aug 2025
- 2025SMART-Ship: A Comprehensive Synchronized Multi-modal Aligned Remote Sensing Targets Dataset and Benchmark for Berthed Ships Analysis4 Aug 2025
- 2025Capturing More: Learning Multi-Domain Representations for Robust Online Handwriting Verification2 Aug 2025
- 2025Jet Image Generation in High Energy Physics Using Diffusion Models1 Aug 2025
- 2025Adversarial Reconstruction Feedback for Robust Fine-grained Generalization29 Jul 2025
- 2025LH2Face: Loss function for Hard High-quality Face30 Jun 2025
- 2025Trident: Detecting Face Forgeries with Adversarial Triplet Learning29 Jun 2025
- 2025Variational Learning of Disentangled Representations20 Jun 2025
- 2025Fine-grained Image Retrieval via Dual-Vision Adaptation19 Jun 2025
- 2025Self-supervised Latent Space Optimization with Nebula Variational Coding2 Jun 2025
50 newest of 449 All 449 (JSON)
- 2022Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognitioncited by 36
- 2022Generalized Knowledge Distillation via Relationship Matchingcited by 12
- 2024DiSCo: LLM Knowledge Distillation for Efficient Sparse Retrieval in Conversational Searchcited by 9
- 2024UDON: Universal Dynamic Online distillatioN for generic image representationscited by 8
- 2023ProS: Facial Omni-Representation Learning via Prototype-based Self-Distillationcited by 3
- 2023Asymmetric Image Retrieval with Cross Model Compatible Ensemblescited by 2
- 2026EviRank: Structured Relevance Evidence for Multimodal Image Re-rankingcited by 2
- 2022Improving Generalization of Metric Learning via Listwise Self-distillationcited by 1
- 2025Unsupervised Transformer Pre-Training for Images: Self-Distillation, Mean Teachers, and Random Cropscited by 0
- 2026EviRank: Structured Relevance Evidence for Multimodal Image Re-ranking21 Aug 2026
- 2025Unsupervised Transformer Pre-Training for Images: Self-Distillation, Mean Teachers, and Random Crops4 Oct 2025
- 2024DiSCo: LLM Knowledge Distillation for Efficient Sparse Retrieval in Conversational Search18 Oct 2024
- 2024UDON: Universal Dynamic Online distillatioN for generic image representations12 Jun 2024
- 2023ProS: Facial Omni-Representation Learning via Prototype-based Self-Distillation3 Nov 2023
- 2023Asymmetric Image Retrieval with Cross Model Compatible Ensembles30 Mar 2023
- 2022Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition23 Jul 2022
- 2022Improving Generalization of Metric Learning via Listwise Self-distillation17 Jun 2022
- 2022Generalized Knowledge Distillation via Relationship Matching4 May 2022
- 2023Image Captioners Are Scalable Vision Learners Toocited by 89
- 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversioncited by 43
- 2025ILIAS: Instance-Level Image retrieval At Scalecited by 30
- 2026TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignmentcited by 13
- 2025Infusing fine-grained visual knowledge to Vision-Language Modelscited by 2
- 2026PointAlign: Feature-Level Alignment Regularization for 3D Vision-Language Modelscited by 2
- 2023Expanding Frozen Vision-Language Models without Retraining: Towards Improved Robot Perceptioncited by 1
- 2025Training-free Conditional Image Embedding Framework Leveraging Large Vision Language Modelscited by 1
- 2026IIR-VLM: In-Context Instance-level Recognition for Large Vision-Language Modelscited by 1
- 2026TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment13 Apr 2026
- 2026PointAlign: Feature-Level Alignment Regularization for 3D Vision-Language Models28 Feb 2026
- 2026IIR-VLM: In-Context Instance-level Recognition for Large Vision-Language Models20 Jan 2026
- 2025Training-free Conditional Image Embedding Framework Leveraging Large Vision Language Models26 Dec 2025
- 2025Infusing fine-grained visual knowledge to Vision-Language Models16 Aug 2025
- 2025ILIAS: Instance-Level Image retrieval At Scale17 Feb 2025
- 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion6 Feb 2025
- 2023Expanding Frozen Vision-Language Models without Retraining: Towards Improved Robot Perception31 Aug 2023
- 2023Image Captioners Are Scalable Vision Learners Too13 Jun 2023
- 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
- 2023DiSK: A Diffusion Model for Structured Knowledgecited by 16
- 2024Imperceptible Protection against Style Imitation from Diffusion Modelscited by 9
- 2025Jet Image Generation in High Energy Physics Using Diffusion Modelscited by 0
- 2025Jet Image Generation in High Energy Physics Using Diffusion Models1 Aug 2025
- 2024Imperceptible Protection against Style Imitation from Diffusion Models28 Mar 2024
- 2023DiSK: A Diffusion Model for Structured Knowledge8 Dec 2023
- 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models7 Dec 2022
- 2025HPS: Hard Preference Sampling for Human Preference Alignment20 Feb 2025
- 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
- 2023A Cookbook of Self-Supervised Learningcited by 250
- 2021NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wildcited by 136
- 2023Hyperbolic Image-Text Representationscited by 136
- 2021Fine-Grained Image Analysis with Deep Learning: A Surveycited by 135
- 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
- 2022CyCLIP: Cyclic Contrastive Language-Image Pretrainingcited by 127
- 2022What to Hide from Your Students: Attention-Guided Masked Image Modelingcited by 118
- 2022Hyperbolic Vision Transformers: Combining Improvements in Metric Learningcited by 111
- 2021ResLT: Residual Learning for Long-tailed Recognitioncited by 96
- 2021Training Vision Transformers for Image Retrievalcited by 90
- 2021ACE: Ally Complementary Experts for Solving Long-Tailed Recognition in One-Shotcited by 86
12 most cited of 423 All 423 (JSON)
- 2026Masked Swingers: Harnessing Data Augmentation to Advance Autoencoders for Self-Supervised Learning29 Sep 2026
- 2026ProtoSeam: Lifting Classifier Training with Latent Gaussian Mixture Models28 Sep 2026
- 2026Multimodal Floorplan Encoding: Learning Dense Modality-Invariant Representations11 Sep 2026
- 2026RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learning2 Sep 2026
- 2026Illuminating Visual Identity in Universal Multimodal Embeddings3 Aug 2026
- 2026Face and Voice Cross-modal Association with Learning Convex Feature Embedding30 Jul 2026
- 2026Understanding Submodular Information Measure Based Objectives for Representation Learning: A Variance and Separation Perspective30 Jul 2026
- 2026COAST: Context-Aware Differential Learning for Gene Expression Prediction in Spatial Transcriptomics10 Jul 2026
- 2026Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings13 Jun 2026
- 2026Learning to Align Generative Appearance Priors for Fine-grained Image Retrieval11 May 2026
- 2026ID-Sim: An Identity-Focused Similarity Metric6 Apr 2026
- 2026ELViS: Efficient Visual Similarity from Local Descriptors that Generalizes Across Domains30 Mar 2026
12 newest of 423 All 423 (JSON)