mlsanityData to 14 Oct 2026

Urania: Differentially Private Insights into AI Use

arXiv 2506.04681 ↗First posted 5 Jun 2025cs.LG cs.AI cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

We introduce Urania, a novel framework for generating insights about LLM chatbot interactions with rigorous differential privacy (DP) guarantees. The framework employs a private clustering mechanism and innovative keyword extraction methods, including frequency-based, TF-IDF-based, and LLM-guided approaches. By leveraging DP tools such as clustering, partition selection, and histogram-based summarization, Urania provides end-to-end privacy protection. Our evaluation assesses lexical and semantic content preservation, pair similarity, and LLM-based metrics, benchmarking against a non-private Clio-inspired pipeline (Tamkin et al., 2024). Moreover, we develop a simple empirical privacy evaluation that demonstrates the enhanced robustness of our DP pipeline. The results show the framework's ability to extract meaningful conversational insights while maintaining stringent user privacy, effectively balancing data utility with privacy preservation.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 15 of its 42 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

15 of 42 references.

References · 15

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  5. 2021Membership Inference Attacks From First Principlescited by 878
  6. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  7. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  8. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 271
  9. 2025Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversationscited by 96
  10. 2024Clio: Privacy-Preserving Insights into Real-World AI Usecited by 72
  11. 2020Differentially Private Clustering: Tight Approximation Ratioscited by 34
  12. 2021FriendlyCore: Practical Differentially Private Aggregationcited by 34
3 more
  1. 2021Locally Private k-Means in One Roundcited by 19
  2. 2023LLM-TAKE: Theme Aware Keyword Extraction Using Large Language Modelscited by 8
  3. 2020Differentially private partition selectioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025AgentCrypt: Advancing Privacy and (Secure) Computation in AI Agent Collaborationcited by 2
  2. 2026Embedding-Aware Feature Discovery: Bridging Latent Representations and Interpretable Features in Event Sequencescited by 1
  3. 2026Probing Memorization of Tabular In-Context Learningcited by 1
  4. 2026PrivCert: Certifying Statement Support under Differential Privacycited by 0