ProfileFoundry: A Synthetic Person-Object Substrate for Privacy, Memory, and Tool-Use Evaluation in LLM Agent
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Foundation-model research increasingly needs data about people: user state, personal histories, relationships, contact-like fields, documents, and longitudinal updates. Real user data is difficult to share, perturb, audit, or redistribute responsibly, while independently generated fake fields rarely preserve the cross-field and temporal consistency needed for controlled evaluation. We present ProfileFoundry, a deterministic generator and fixed reference release of 100,000 adult synthetic Person Objects across eight locales. Each object combines a typed current snapshot, household, family, and employer links, snapshot-aligned events, normalized relational views, and generation provenance. The release contains 709,228 events, 40,338 households, 52,491 employers, and 518,564 directed relationship edges. We report evidence in separate categories: selected population-marginal comparisons, per-object invariant checks, release-wide referential and temporal closure, and coincidence/provenance screens. A pilot case study, MatchDesk, uses certified coincidences, typed events, and history truncation to evaluate whether models distinguish corroborated identity evidence from underdetermined matches. ProfileFoundry is not a population-fidelity model, a rendered-text corpus, or a formal privacy mechanism. Instead, it is a responsible synthetic source layer for constructing downstream foundation-model evaluations involving memory, privacy, document understanding, record linkage, and agent state while keeping the synthetic person behind each artifact inspectable.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 14 of its 34 references that are in the map. First posted on arXiv 24 Jun 2026.
Builds on
14 of 34 references.
References · 14
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
- 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
- 2023LaMP: When Large Language Models Meet Personalizationcited by 437
- 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
- 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 178
- 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175
- 2024A Synthetic Dataset for Personal Attribute Inferencecited by 36
- 2025PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Datacited by 27
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DynamicMem: A Long-Horizon Memory Benchmark in Real-World Settings
- 2026PersonaTrace: Synthesizing Realistic Digital Footprints with LLM Agents
- 2025Automated Profile Inference with Language Model Agents
- 2025An Empirical Study of Validating Synthetic Data for Text-Based Person Retrieval
- 2026Profiling for Pennies: Unveiling the Privacy Iceberg of LLM Agents
- 2025PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalization
- 2025Auditing M-LLMs for Privacy Risks: A Synthetic Benchmark and Evaluation Framework ↗
- 2026Behaviorally Grounded User Profiles from the Wild for Personalized Alignment and Multi-Perspective Reasoning