# Panza: Design and Analysis of a Fully-Local Personalized Text Writing Assistant

arXiv: https://arxiv.org/abs/2407.10994 · HTML: https://mlsanity.com/paper/2407.10994/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [RAFT: Adapting Language Model to Domain Specific RAG](https://mlsanity.com/paper/2403.10131/) (2024)
- [Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning](https://mlsanity.com/paper/2402.04401/) (2024)
- [RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation](https://mlsanity.com/paper/2401.04679/) (2024)
- [LongForm: Effective Instruction Tuning with Reverse Instructions](https://mlsanity.com/paper/2304.08460/) (2023)

## Built on by (most cited first, 5 of 5)

- [Do LLMs Make Mistakes Like Students? Exploring Natural Alignment between Language Models and Human Error Patterns](https://mlsanity.com/paper/2502.15140/) (2025)
- [PersonalBench: Measuring the Authorship Gap in LLM Personalization](https://mlsanity.com/paper/2608.19746/) (2026)
- [Building Persona-Based Agents On Demand: Tailoring Multi-Agent Workflows to User Needs](https://mlsanity.com/paper/2604.27882/) (2026)
- [CASPER in the Machine: Insights into Character Variety in LLM-Generated Stories](https://mlsanity.com/paper/2606.22454/) (2026)
- [Small Is Enough: Per-User Style Rewriting of AI-Edited Text via LoRA Adapters](https://mlsanity.com/paper/2607.29238/) (2026)

## Abstract (arXiv)

The availability of powerful open-source large language models (LLMs) opens exciting use-cases, such as using personal data to fine-tune these models to imitate a user's unique writing style. Two key requirements for such assistants are personalization - in the sense that the assistant should recognizably reflect the user's own writing style - and privacy - users may justifiably be wary of uploading extremely personal data, such as their email archive, to a third-party service. In this paper, we present a new design and evaluation for such an automated assistant, for the specific use case of email generation, which we call Panza. Panza's personalization features are based on a combination of fine-tuning using a variant of the Reverse Instructions technique together with Retrieval-Augmented Generation (RAG). We demonstrate that this combination allows us to fine-tune an LLM to reflect a user's writing style using limited data, while executing on extremely limited resources, e.g. on a free Google Colab instance. Our key methodological contribution is the first detailed study of evaluation metrics for this personalized writing task, and of how different choices of system components--the use of RAG and of different fine-tuning approaches-impact the system's performance. Additionally, we demonstrate that very little data - under 100 email samples - are sufficient to create models that convincingly imitate humans. This finding showcases a previously-unknown attack vector in language models - that access to a small number of writing samples can allow a bad actor to cheaply create generative models that imitate a target's writing style. We are releasing the full Panza code as well as three new email datasets licensed for research use at https://github.com/IST-DASLab/PanzaMail.