The pretraining domain outweighs the training objective in setting the privacy-utility trade-off of differentially private medical image analysis

Differential privacy limits how much each training image can influence a medical imaging model but lowers its diagnostic accuracy. Pretrained weights reduce this loss. It is unknown whether the pretraining objective or domain improves accuracy more. Here we train chest radiograph classifiers under differential privacy from five starting points on 591,884 radiographs from five datasets. We show that supervised pretraining on chest radiographs ranks first in 24 of 25 combinations of dataset and privacy setting. Measured by the area under the receiver operating characteristic curve, its lead over supervised pretraining on natural images is 2.6% without privacy and 15.3% at the strictest budget. From that starting point, changing the domain improves this area 2.2 to 3.7 times more than changing the objective. Private pretraining on chest radiographs lowers this area by about 5% under privacy but outperforms the three starting points without patient data at the strictest budget. Under privacy, supervised chest radiograph pretraining leads in the worst-performing demographic subgroup.