# Contrastive-Adversarial and Diffusion: Exploring pre-training and fine-tuning strategies for sulcal identification

arXiv: https://arxiv.org/abs/2405.19204 · HTML: https://mlsanity.com/paper/2405.19204/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 29 May 2024.

## Builds on (most cited first, 14 of 14)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesis](https://mlsanity.com/paper/2303.14081/) (2023)
- [DiffMix: Diffusion Model-based Data Synthesis for Nuclei Segmentation and Classification in Imbalanced Pathology Image Datasets](https://mlsanity.com/paper/2306.14132/) (2023)
- [Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction](https://mlsanity.com/paper/2308.10157/) (2023)
- [ASCON: Anatomy-aware Supervised Contrastive Learning Framework for Low-dose CT Denoising](https://mlsanity.com/paper/2307.12225/) (2023)
- [Domain Adaptation for Medical Image Segmentation using Transformation-Invariant Self-Training](https://mlsanity.com/paper/2307.16660/) (2023)
- [Annotator Consensus Prediction for Medical Image Segmentation with Diffusion Models](https://mlsanity.com/paper/2306.09004/) (2023)
- [EPVT: Environment-aware Prompt Vision Transformer for Domain Generalization in Skin Lesion Recognition](https://mlsanity.com/paper/2304.01508/) (2023)
- [Semi-supervised Domain Adaptive Medical Image Segmentation through Consistency Regularized Disentangled Contrastive Learning](https://mlsanity.com/paper/2307.02798/) (2023)
- [Faithful Synthesis of Low-dose Contrast-enhanced Brain MRI Scans using Noise-preserving Conditional GANs](https://mlsanity.com/paper/2306.14678/) (2023)
- [Self-supervised learning via inter-modal reconstruction and feature projection networks for label-efficient 3D-to-2D segmentation](https://mlsanity.com/paper/2307.03008/) (2023)
- [Improved Multi-Shot Diffusion-Weighted MRI with Zero-Shot Self-Supervised Learning Reconstruction](https://mlsanity.com/paper/2308.05103/) (2023)

## Abstract (arXiv)

In the last decade, computer vision has witnessed the establishment of various training and learning approaches. Techniques like adversarial learning, contrastive learning, diffusion denoising learning, and ordinary reconstruction learning have become standard, representing state-of-the-art methods extensively employed for fully training or pre-training networks across various vision tasks. The exploration of fine-tuning approaches has emerged as a current focal point, addressing the need for efficient model tuning with reduced GPU memory usage and time costs while enhancing overall performance, as exemplified by methodologies like low-rank adaptation (LoRA). Key questions arise: which pre-training technique yields optimal results - adversarial, contrastive, reconstruction, or diffusion denoising? How does the performance of these approaches vary as the complexity of fine-tuning is adjusted? This study aims to elucidate the advantages of pre-training techniques and fine-tuning strategies to enhance the learning process of neural networks in independent identical distribution (IID) cohorts. We underscore the significance of fine-tuning by examining various cases, including full tuning, decoder tuning, top-level tuning, and fine-tuning of linear parameters using LoRA. Systematic summaries of model performance and efficiency are presented, leveraging metrics such as accuracy, time cost, and memory efficiency. To empirically demonstrate our findings, we focus on a multi-task segmentation-classification challenge involving the paracingulate sulcus (PCS) using different 3D Convolutional Neural Network (CNN) architectures by using the TOP-OSLO cohort comprising 596 subjects.