# Accelerating Private Large Transformers Inference through Fine-grained Collaborative Computation

arXiv: https://arxiv.org/abs/2412.16537 · HTML: https://mlsanity.com/paper/2412.16537/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2024.

## Built on by (most cited first, 1 of 1)

- [Privacy-Preserving Inference for Quantized BERT Models](https://mlsanity.com/paper/2508.01636/) (2025)

## Abstract (arXiv)

Homomorphic encryption (HE) and secret sharing (SS) enable computations on encrypted data, providing significant privacy benefits for large transformer-based models (TBM) in sensitive sectors like medicine and finance. However, private TBM inference incurs significant costs due to the coarse-grained application of HE and SS. We present FASTLMPI, a new approach to accelerate private TBM inference through fine-grained computation optimization. Specifically, through the fine-grained co-design of homomorphic encryption and secret sharing, FASTLMPI achieves efficient protocols for matrix multiplication, SoftMax, LayerNorm, and GeLU. In addition, FASTLMPI introduces a precise segmented approximation technique for differentiable non-linear, improving its fitting accuracy while maintaining a low polynomial degree. Compared to solution BOLT (S&P'24), FASTLMPI shows a remarkable 54% to 64% decrease in runtime and an impressive 72.2% reduction in communication costs.