mlsanityData to 7 Oct 2026

PriViT: Vision Transformers for Fast Private Inference

arXiv 2310.04604 ↗First posted 6 Oct 2023cs.CR cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

The Vision Transformer (ViT) architecture has emerged as the backbone of choice for state-of-the-art deep models for computer vision applications. However, ViTs are ill-suited for private inference using secure multi-party computation (MPC) protocols, due to the large number of non-polynomial operations (self-attention, feed-forward rectifiers, layer normalization). We propose PriViT, a gradient based algorithm to selectively "Taylorize" nonlinearities in ViTs while maintaining their prediction accuracy. Our algorithm is conceptually simple, easy to implement, and achieves improved performance over existing approaches for designing MPC-friendly transformer architectures in terms of achieving the Pareto frontier in latency-accuracy. We confirm these improvements via experiments on several standard image classification tasks. Public code is available at https://github.com/NYU-DICE-Lab/privit.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 25 of its 45 references that are in the map. First posted on arXiv 6 Oct 2023.

Builds on

25 of 45 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2016Layer Normalizationcited by 5,579
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  7. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  8. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  9. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  10. 2017Random Erasing Data Augmentationcited by 1,235
  11. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  12. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
13 more
  1. 2022cosFormer: Rethinking Softmax in Attentioncited by 212
  2. 2021SOFT: Softmax-free Transformer with Linear Complexitycited by 128
  3. 2022MPCFormer: fast, performant and private Transformer inference with MPCcited by 67
  4. 2021DeepReDuce: ReLU Reduction for Fast Private Inferencecited by 50
  5. 2020CryptoNAS: Private Inference on a ReLU Budgetcited by 40
  6. 2022Selective Network Linearization for Efficient Private Inferencecited by 31
  7. 2023Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inferencecited by 20
  8. 2021Circa: Stochastic ReLUs for Private Deep Learningcited by 16
  9. 2021Sphynx: ReLU-Efficient Network Design for Private Inferencecited by 15
  10. 2023RRNet: Towards ReLU-Reduced Neural Network for Two-party Computation Based Private Inferencecited by 12
  11. 2021UFO-ViT: High Performance Linear Vision Transformer without Softmaxcited by 11
  12. 2022HAAC: A Hardware-Software Co-Design to Accelerate Garbled Circuitscited by 5
  13. 2023Towards Fast and Scalable Private Inferencecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024AERO: Entropy-Guided Framework for Private LLM Inferencecited by 3
  2. 2024TruncFormer: Private LLM Inference Using Only Truncationscited by 1
  3. 2025CrypTorch: PyTorch-based Auto-tuning Compiler for Machine Learning with Multi-party Computationcited by 1
  4. 2026UFO: Unlocking Ultra-Efficient Quantized Private Inference with Protocol and Algorithm Co-Optimizationcited by 0