# LEAP: LLM-powered End-to-end Automatic Library for Processing Social Science Queries on Unstructured Data

arXiv: https://arxiv.org/abs/2501.03892 · HTML: https://mlsanity.com/paper/2501.03892/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2025.

## Built on by (most cited first, 7 of 7)

- [REPRO-Bench: Can Agentic AI Systems Assess the Reproducibility of Social Science Research?](https://mlsanity.com/paper/2507.18901/) (2025)
- [ScaleDoc: Scaling LLM-based Predicates over Large Document Collections](https://mlsanity.com/paper/2509.12610/) (2025)
- [Multi-Segment Attention: Enabling Efficient KV-Cache Management for Faster Large Language Model Serving](https://mlsanity.com/paper/2606.02964/) (2026)
- [LoopServe: An Adaptive Dual-phase LLM Inference Acceleration System for Multi-Turn Dialogues](https://mlsanity.com/paper/2507.13681/) (2025)
- [DRAMA: Unifying Data Retrieval and Analysis for Open-Domain Analytic Queries](https://mlsanity.com/paper/2510.27238/) (2025)
- [SODIUM: From Open Web Data to Queryable Databases](https://mlsanity.com/paper/2603.18447/) (2026)
- [BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence](https://mlsanity.com/paper/2609.20886/) (2026)

## Abstract (arXiv)

Social scientists are increasingly interested in analyzing the semantic information (e.g., emotion) of unstructured data (e.g., Tweets), where the semantic information is not natively present. Performing this analysis in a cost-efficient manner requires using machine learning (ML) models to extract the semantic information and subsequently analyze the now structured data. However, this process remains challenging for domain experts. To demonstrate the challenges in social science analytics, we collect a dataset, QUIET-ML, of 120 real-world social science queries in natural language and their ground truth answers. Existing systems struggle with these queries since (1) they require selecting and applying ML models, and (2) more than a quarter of these queries are vague, making standard tools like natural language to SQL systems unsuited. To address these issues, we develop LEAP, an end-to-end library that answers social science queries in natural language with ML. LEAP filters vague queries to ensure that the answers are deterministic and selects from internally supported and user-defined ML functions to extend the unstructured data to structured tables with necessary annotations. LEAP further generates and executes code to respond to these natural language queries. LEAP achieves a 100% pass @ 3 and 92% pass @ 1 on QUIET-ML, with a 1.06 average end-to-end cost, of which code generation costs 0.02.