mlsanityData to 7 Oct 2026

Input Reconstruction Attack against Vertical Federated Large Language Models

arXiv 2311.07585 ↗First posted 7 Nov 2023cs.CL cs.AI cs.CRcited by 5

Not on the map. Why?

Abstract (arXiv)

Recently, large language models (LLMs) have drawn extensive attention from academia and the public, due to the advent of the ChatGPT. While LLMs show their astonishing ability in text generation for various tasks, privacy concerns limit their usage in real-life businesses. More specifically, either the user's inputs (the user sends the query to the model-hosting server) or the model (the user downloads the complete model) itself will be revealed during the usage. Vertical federated learning (VFL) is a promising solution to this kind of problem. It protects both the user's input and the knowledge of the model by splitting the model into a bottom part and a top part, which is maintained by the user and the model provider, respectively. However, in this paper, we demonstrate that in LLMs, VFL fails to protect the user input since it is simple and cheap to reconstruct the input from the intermediate embeddings. Experiments show that even with a commercial GPU, the input sentence can be reconstructed in only one second. We also discuss several possible solutions to enhance the privacy of vertical federated LLMs.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 17 of its 23 references that are in the map. First posted on arXiv 7 Nov 2023.

Builds on

17 of 23 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2016Deep Learning with Differential Privacycited by 2,682
  7. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  8. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  9. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  10. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
  11. 2020Information Leakage in Embedding Modelscited by 199
  12. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 189
5 more
  1. 2020Feature Inference Attack on Model Predictions in Vertical Federated Learningcited by 105
  2. 2020NoPeek: Information leakage reduction to share activations in distributed deep learningcited by 66
  3. 2021UnSplit: Data-Oblivious Model Inversion, Model Stealing, and Label Inference Attacks Against Split Learningcited by 52
  4. 2023PUMA: Secure Inference of LLaMA-7B in Five Minutescited by 44
  5. 2023Long Text and Multi-Table Summarization: Dataset and Methodcited by 23

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  2. 2024Vertical Federated Learning for Effectiveness, Security, Applicability: A Surveycited by 14
  3. 2025Depth Gives a False Sense of Privacy: LLM Internal States Inversioncited by 14
  4. 2025Vertical Federated Learning in Practice: The Good, the Bad, and the Uglycited by 4
  5. 2025GPT, But Backwards: Exactly Inverting Language Model Outputscited by 2