# Measure-Observe-Remeasure: An Interactive Paradigm for Differentially-Private Exploratory Analysis

arXiv: https://arxiv.org/abs/2406.01964 · HTML: https://mlsanity.com/paper/2406.01964/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2024.

## Built on by (most cited first, 24 of 24)

- [Security Concerns for Large Language Models: A Survey](https://mlsanity.com/paper/2505.18889/) (2025)
- [When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs](https://mlsanity.com/paper/2410.14569/) (2024)
- [Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks](https://mlsanity.com/paper/2501.16466/) (2025)
- [Large Language Model (LLM) for Software Security: Code Analysis, Malware Analysis, Reverse Engineering](https://mlsanity.com/paper/2504.07137/) (2025)
- [The Dark Side of Trust: Authority Citation-Driven Jailbreak Attacks on Large Language Models](https://mlsanity.com/paper/2411.11407/) (2024)
- [Applying Refusal-Vector Ablation to Llama 3.1 70B Agents](https://mlsanity.com/paper/2410.10871/) (2024)
- [GAVEL: Towards Rule-Based Safety Through Activation Monitoring](https://mlsanity.com/paper/2601.19768/) (2026)
- [On the Ethical Considerations of Generative Agents](https://mlsanity.com/paper/2411.19211/) (2024)
- [StructTransform: A Scalable Attack Surface for Safety-Aligned Large Language Models](https://mlsanity.com/paper/2502.11853/) (2025)
- [Advancing Machine-Generated Text Detection from an Easy to Hard Supervision Perspective](https://mlsanity.com/paper/2511.00988/) (2025)
- [PreScam: A Benchmark for Predicting Scam Progression from Early Conversations](https://mlsanity.com/paper/2605.12243/) (2026)
- [Is On-Device AI Broken and Exploitable? Assessing the Trust and Ethics in Small Language Models](https://mlsanity.com/paper/2406.05364/) (2024)
- [Unintentional Consequences: Generative AI Use for Cybercrime](https://mlsanity.com/paper/2505.23733/) (2025)
- [Phishing Detection in the Gen-AI Era: Quantized LLMs vs Classical Models](https://mlsanity.com/paper/2507.07406/) (2025)
- [PiMRef: Deducing Ever-evolving Spear-phishing Emails with Knowledge Base Invariants](https://mlsanity.com/paper/2507.15393/) (2025)
- [UNSEEN: A Cross-Stack LLM Unlearning Defense against AR-LLM Social Engineering Attacks](https://mlsanity.com/paper/2604.23141/) (2026)
- [PhySE: A Psychological Framework for Real-Time AR-LLM Social Engineering Attacks](https://mlsanity.com/paper/2604.23148/) (2026)
- [SoK: Intent-Oriented Systematization of Multi-Turn LLM Jailbreaks](https://mlsanity.com/paper/2608.01117/) (2026)
- [Analysing Safety Risks in LLMs Fine-Tuned with Pseudo-Malicious Cyber Security Data](https://mlsanity.com/paper/2505.09974/) (2025)
- [MultiPhishGuard: An Explainable and Adaptive Multi-Agent LLM System for Phishing Email Detection](https://mlsanity.com/paper/2505.23803/) (2025)
- [Jailbreaking Generative AI: Multivector Phishing Threats and Transformer based Defenses](https://mlsanity.com/paper/2507.12185/) (2025)
- [Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats](https://mlsanity.com/paper/2606.26377/) (2026)
- [The Fragility of Trigger-Tag Mechanisms for Misuse Detection in Open-Weight LLMs](https://mlsanity.com/paper/2610.03124/) (2026)
- [Beware EviLLM: Enabling Vulnerability Injection via Large Language Models](https://mlsanity.com/paper/2610.03857/) (2026)

## Abstract (arXiv)

Differential privacy (DP) has the potential to enable privacy-preserving analysis on sensitive data, but requires analysts to judiciously spend a limited ``privacy loss budget'' ε across queries. Analysts conducting exploratory analyses do not, however, know all queries in advance and seldom have DP expertise. Thus, they are limited in their ability to specify ε allotments across queries prior to an analysis. To support analysts in spending ε efficiently, we propose a new interactive analysis paradigm, Measure-Observe-Remeasure, where analysts ``measure'' the database with a limited amount of ε, observe estimates and their errors, and remeasure with more ε as needed. We instantiate the paradigm in an interactive visualization interface which allows analysts to spend increasing amounts of ε under a total budget. To observe how analysts interact with the Measure-Observe-Remeasure paradigm via the interface, we conduct a user study that compares the utility of ε allocations and findings from sensitive data participants make to the allocations and findings expected of a rational agent who faces the same decision task. We find that participants are able to use the workflow relatively successfully, including using budget allocation strategies that maximize over half of the available utility stemming from ε allocation. Their loss in performance relative to a rational agent appears to be driven more by their inability to access information and report it than to allocate ε.