# Fortify Your Foundations: Practical Privacy and Security for Foundation Model Deployments In The Cloud

arXiv: https://arxiv.org/abs/2410.05930 · HTML: https://mlsanity.com/paper/2410.05930/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 67 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [NExT-GPT: Any-to-Any Multimodal LLM](https://mlsanity.com/paper/2309.05519/) (2023)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks](https://mlsanity.com/paper/2309.17410/) (2023)
- [A Survey on Homomorphic Encryption Schemes: Theory and Implementation](https://mlsanity.com/paper/1704.03578/) (2017)
- [Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis](https://mlsanity.com/paper/1802.09941/) (2018)
- [CrypTen: Secure Multi-Party Computation Meets Machine Learning](https://mlsanity.com/paper/2109.00984/) (2021)
- [Foundational Models Defining a New Era in Vision: A Survey and Outlook](https://mlsanity.com/paper/2307.13721/) (2023)
- [Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware](https://mlsanity.com/paper/1806.03287/) (2018)
- [DAWN: Dynamic Adversarial Watermarking of Neural Networks](https://mlsanity.com/paper/1906.00830/) (2019)
- [The Cost of Training NLP Models: A Concise Overview](https://mlsanity.com/paper/2004.08900/) (2020)
- [The Efficiency Misnomer](https://mlsanity.com/paper/2110.12894/) (2021)
- [Privacy-Preserving Machine Learning with Fully Homomorphic Encryption for Deep Neural Network](https://mlsanity.com/paper/2106.07229/) (2021)
- [A Systematic Review on Model Watermarking for Neural Networks](https://mlsanity.com/paper/2009.12153/) (2020)
- [Intellectual Property Protection for Deep Learning Models: Taxonomy, Methods, Attacks, and Evaluations](https://mlsanity.com/paper/2011.13564/) (2020)
- [Machine Learning with Confidential Computing: A Systematization of Knowledge](https://mlsanity.com/paper/2208.10134/) (2022)
- [Conflicting Interactions Among Protection Mechanisms for Machine Learning Models](https://mlsanity.com/paper/2207.01991/) (2022)
- [Intel TDX Demystified: A Top-Down Approach](https://mlsanity.com/paper/2303.15540/) (2023)
- [On the Impact of Memory Allocation on High-Performance Query Processing](https://mlsanity.com/paper/1905.01135/) (2019)
- [Privacy-Preserving Machine Learning in Untrusted Clouds Made Simple](https://mlsanity.com/paper/2009.04390/) (2020)

## Built on by (most cited first, 4 of 4)

- [Atlas: A Framework for ML Lifecycle Provenance & Transparency](https://mlsanity.com/paper/2502.19567/) (2025)
- [Confidential LLM Inference: Performance and Cost Across CPU and GPU TEEs](https://mlsanity.com/paper/2509.18886/) (2025)
- [Scalable GPU-Based Integrity Verification for Large Machine Learning Models](https://mlsanity.com/paper/2510.23938/) (2025)
- [PRAG: End-to-End Privacy-Preserving Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.26525/) (2026)

## Abstract (arXiv)

Foundation Models (FMs) display exceptional performance in tasks such as natural language processing and are being applied across a growing range of disciplines. Although typically trained on large public datasets, FMs are often fine-tuned or integrated into Retrieval-Augmented Generation (RAG) systems, which rely on private data. This access, along with their size and costly training, heightens the risk of intellectual property theft. Moreover, multimodal FMs may expose sensitive information. In this work, we examine the FM threat model and discuss the practicality and comprehensiveness of various approaches for securing against them, such as ML-based methods and trusted execution environments (TEEs). We demonstrate that TEEs offer an effective balance between strong security properties, usability, and performance. Specifically, we present a solution achieving less than 10% overhead versus bare metal for the full Llama2 7B and 13B inference pipelines running inside intel SGX and intel TDX. We also share our configuration files and insights from our implementation. To our knowledge, our work is the first to show the practicality of TEEs for securing FMs.