mlsanityData to 7 Oct 2026

SVMax: A Feature Embedding Regularizer

arXiv 2103.02770 ↗First posted 4 Mar 2021cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

A neural network regularizer (e.g., weight decay) boosts performance by explicitly penalizing the complexity of a network. In this paper, we penalize inferior network activations -- feature embeddings -- which in turn regularize the network's weights implicitly. We propose singular value maximization (SVMax) to learn a more uniform feature embedding. The SVMax regularizer supports both supervised and unsupervised learning. Our formulation mitigates model collapse and enables larger learning rates. We evaluate the SVMax regularizer using both retrieval and generative adversarial networks. We leverage a synthetic mixture of Gaussians dataset to evaluate SVMax in an unsupervised setting. For retrieval networks, SVMax achieves significant improvement margins across various ranking losses. Code available at https://bit.ly/3jNkgDt

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 51 of its 64 references that are in the map. First posted on arXiv 4 Mar 2021.

Builds on

51 of 64 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2014Going Deeper with Convolutionscited by 4,231
  10. 2016Improved Techniques for Training GANscited by 2,938
  11. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  12. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
38 more
  1. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  2. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  3. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  4. 2019Contrastive Multiview Codingcited by 1,297
  5. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  6. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  7. 2016Colorful Image Colorizationcited by 1,113
  8. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  9. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 784
  10. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  11. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  12. 2014Deep metric learning using Triplet networkcited by 421
  13. 2016Adversarial Feature Learningcited by 350
  14. 2017Sampling Matters in Deep Embedding Learningcited by 285
  15. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  16. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  17. 2016Unrolled Generative Adversarial Networkscited by 208
  18. 2017VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learningcited by 199
  19. 2020Proxy Anchor Loss for Deep Metric Learningcited by 195
  20. 2019Mode Seeking Generative Adversarial Networks for Diverse Image Synthesiscited by 138
  21. 2019Robust Learning with Jacobian Regularizationcited by 127
  22. 2017Representation Learning by Learning to Countcited by 120
  23. 2018Spreading vectors for similarity searchcited by 114
  24. 2017Deep Metric Learning with Angular Losscited by 107
  25. 2017A Generative Adversarial Approach for Zero-Shot Learning from Noisy Textscited by 96
  26. 2018The Singular Values of Convolutional Layerscited by 95
  27. 2018Triplet-Center Loss for Multi-View 3D Object Retrievalcited by 69
  28. 2019Improved Embeddings with Easy Positive Triplet Miningcited by 66
  29. 2015Metric Learning with Adaptive Density Discriminationcited by 40
  30. 2018Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterizationcited by 35
  31. 2017Learning Spread-out Local Feature Descriptorscited by 34
  32. 2015Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functionscited by 33
  33. 2015Training Deep Networks with Structured Layers by Matrix Backpropagationcited by 28
  34. 2018MGGAN: Solving Mode Collapse using Manifold Guided Trainingcited by 22
  35. 2019Energy Confused Adversarial Metric Learning for Zero-Shot Image Retrieval and Clusteringcited by 10
  36. 2019Boosting Standard Classification Architectures Through a Ranking Regularizercited by 7
  37. 2016The Computational Complexity of Dualitycited by 3
  38. 2017Trace norm regularization and faster inference for embedded speech recognition RNNscited by 3

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025RAE: A Neural Network Dimensionality Reduction Method for Nearest Neighbors Preservation in Vector Search
  2. 2026WRF4CIR: Weight-Regularized Fine-Tuning Network for Composed Image Retrieval
  3. 2024NUDGE: Lightweight Non-Parametric Fine-Tuning of Embeddings for Retrieval
  4. 2024Singular Value Scaling: Efficient Generative Model Compression via Pruned Weights Refinement
  5. 2025Beyond instruction-conditioning, MoTE: Mixture of Task Experts for Multi-task Embedding Models
  6. 2026Generative Retrieval via Diffusion Transformer with Metric-Ordered Sequence Training and Hybrid-Policy Preference Optimization
  7. 2023Loss Functions and Metrics in Deep Learning
  8. 2025Optimizing What Matters: AUC-Driven Learning for Robust Neural Retrieval