mlsanityData to 14 Oct 2026

Differentially Private CutMix for Split Learning with Vision Transformer

arXiv 2210.15986 ↗First posted 28 Oct 2022cs.DC cs.CV cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Recently, vision transformer (ViT) has started to outpace the conventional CNN in computer vision tasks. Considering privacy-preserving distributed learning with ViT, federated learning (FL) communicates models, which becomes ill-suited due to ViT' s large model size and computing costs. Split learning (SL) detours this by communicating smashed data at a cut-layer, yet suffers from data privacy leakage and large communication costs caused by high similarity between ViT' s smashed data and input data. Motivated by this problem, we propose DP-CutMixSL, a differentially private (DP) SL framework by developing DP patch-level randomized CutMix (DP-CutMix), a novel privacy-preserving inter-client interpolation scheme that replaces randomly selected patches in smashed data. By experiment, we show that DP-CutMixSL not only boosts privacy guarantees and communication efficiency, but also achieves higher accuracy than its Vanilla SL counterpart. Theoretically, we analyze that DP-CutMix amplifies R'enyi DP (RDP), which is upper-bounded by its Vanilla Mixup counterpart.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 28 Oct 2022.

Builds on

29 of 38 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  8. 2019Advances and Open Problems in Federated Learningcited by 3,046
  9. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  10. 2016Deep Learning with Differential Privacycited by 2,728
  11. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  12. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
17 more
  1. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  2. 2017Renyi Differential Privacycited by 650
  3. 2021Intriguing Properties of Vision Transformerscited by 446
  4. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 395
  5. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  6. 2020SplitFed: When Federated Learning Meets Split Learningcited by 310
  7. 2018Distributed learning of deep neural network over multiple agentscited by 279
  8. 2020FedVision: An Online Visual Object Detection Platform Powered by Federated Learningcited by 124
  9. 2018Wireless Network Intelligence at the Edgecited by 92
  10. 2019Detailed comparison of communication efficiency of split learning and federated learningcited by 77
  11. 2021DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentationscited by 34
  12. 2021FedCV: A Federated Learning Framework for Diverse Computer Vision Taskscited by 32
  13. 2022Differentially Private Label Protection in Split Learningcited by 26
  14. 2021Evaluation and Optimization of Distributed Machine Learning Techniques for Internet of Thingscited by 20
  15. 2022Feature Space Hijacking Attacks against Differentially Private Split Learningcited by 15
  16. 2022Visual Transformer Meets CutMix for Improved Accuracy, Communication Efficiency, and Data Privacy in Split Learningcited by 8
  17. 2021AirMixML: Over-the-Air Data Mixup for Inherently Privacy-Preserving Edge Machine Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 4; 2026: 1.

13 papers.

Sort:

By area:

  1. 2025Paper2Poster: Towards Multimodal Poster Automation from Scientific Paperscited by 63
  2. 2025PosterGen: Aesthetic-Aware Multi-Modal Paper-to-Poster Generation via Multi-Agent LLMscited by 11
  3. 2023DP-Mix: Mixup-based Data Augmentation for Differentially Private Learningcited by 9
  4. 2023FeSViBS: Federated Split Learning of Vision Transformer with Block Samplingcited by 7
  5. 2024A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspectivecited by 7
  6. 2024Efficient Partitioning Vision Transformer on Edge Devices for Distributed Inferencecited by 6
  7. 2023SLPerf: a Unified Framework for Benchmarking Split Learningcited by 3
  8. 2024Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMixcited by 2
  9. 2023SplitAMC: Split Learning for Robust Automatic Modulation Classificationcited by 1
  10. 2025Fine-tuning Multimodal Transformers on Edge: A Parallel Split Learning Approachcited by 1
  11. 2024Enhancing Privacy in ControlNet and Stable Diffusion via Split Learningcited by 0
  12. 2025MedSegNet10: A Publicly Accessible Network Repository for Split Federated Medical Image Segmentationcited by 0
1 more
  1. 2026Scalable and Private Federated Learning Using Distributed Differential Privacy and Secure Aggregationcited by 0