mlsanityData to 14 Oct 2026

DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learning

arXiv 2309.11782 ↗First posted 21 Sep 2023cs.CV cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) has gained remarkable success, for which contrastive learning (CL) plays a key role. However, the recent development of new non-CL frameworks has achieved comparable or better performance with high improvement potential, prompting researchers to enhance these frameworks further. Assimilating CL into non-CL frameworks has been thought to be beneficial, but empirical evidence indicates no visible improvements. In view of that, this paper proposes a strategy of performing CL along the dimensional direction instead of along the batch direction as done in conventional contrastive learning, named Dimensional Contrastive Learning (DimCL). DimCL aims to enhance the feature diversity, and it can serve as a regularizer to prior SSL frameworks. DimCL has been found to be effective, and the hardness-aware property is identified as a critical reason for its success. Extensive experimental results reveal that assimilating DimCL into SSL frameworks leads to performance improvement by a non-trivial margin on various datasets and backbone architectures.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 49 of its 71 references that are in the map. First posted on arXiv 21 Sep 2023.

Builds on

49 of 71 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  11. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  12. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
37 more
  1. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  2. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  3. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  4. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  5. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  6. 2019Contrastive Multiview Codingcited by 1,301
  7. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  9. 2021Scaling Vision Transformerscited by 999
  10. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  11. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  12. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 786
  13. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  14. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  15. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  16. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  17. 2020Understanding the Behaviour of Contrastive Losscited by 398
  18. 2020Debiased Contrastive Learningcited by 356
  19. 2021XCiT: Cross-Covariance Image Transformerscited by 338
  20. 2020Whitening for Self-Supervised Representation Learningcited by 279
  21. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  22. 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 200
  23. 2020Learning Robust Representations via Multi-View Information Bottleneckcited by 191
  24. 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
  25. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  26. 2021Solo-learn: A Library of Self-supervised Methods for Visual Representation Learningcited by 116
  27. 2020BYOL works even without batch statisticscited by 95
  28. 2020Contrastive Learning with Adversarial Examplescited by 91
  29. 2019Hierarchical Clustering with Hard-batch Triplet Loss for Person Re-identificationcited by 83
  30. 2021Weakly Supervised Contrastive Learningcited by 81
  31. 2020Conditional Negative Sampling for Contrastive Learning of Visual Representationscited by 57
  32. 2021Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learningcited by 47
  33. 2021Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translationcited by 45
  34. 2018Mining on Manifolds: Metric Learning without Labelscited by 36
  35. 2020DC-BERT: Decoupling Question and Document for Efficient Contextual Encodingcited by 20
  36. 2021Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Modelcited by 16
  37. 2021LoOp: Looking for Optimal Hard Negative Embeddings for Deep Metric Learningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 1; 2026: 2.

6 papers.

  1. 2025One-Step Flow Q-Learning: Addressing the Diffusion Policy Bottleneck in Offline Reinforcement Learningcited by 11
  2. 2024Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defensescited by 7
  3. 2024Generating Human Understandable Explanations for Node Embeddingscited by 2
  4. 2023From Pretext to Purpose: Batch-Adaptive Self-Supervised Learningcited by 1
  5. 2026Compositional Average Velocity Modeling for Efficient Diffusion Offline Reinforcement Learningcited by 0
  6. 2026Metric-Driven Reinforcement Learning for Object Hallucination Reduction in LVLM-Based Image Captioningcited by 0