# FlashProfile: A Framework for Synthesizing Data Profiles

arXiv: https://arxiv.org/abs/1709.05725 · HTML: https://mlsanity.com/paper/1709.05725/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2017.

## Built on by (most cited first, 5 of 5)

- [Auto-Validate: Unsupervised Data Validation Using Data-Domain Patterns Inferred from Data Lakes](https://mlsanity.com/paper/2104.04659/) (2021)
- [DataVinci: Learning Syntactic and Semantic String Repairs](https://mlsanity.com/paper/2308.10922/) (2023)
- [Auto-Tag: Tagging-Data-By-Example in Data Lakes](https://mlsanity.com/paper/2112.06049/) (2021)
- [Neural Program Synthesis with Query](https://mlsanity.com/paper/2205.07857/) (2022)
- [Solving Data-centric Tasks using Large Language Models](https://mlsanity.com/paper/2402.11734/) (2024)

## Abstract (arXiv)

We address the problem of learning a syntactic profile for a collection of strings, i.e. a set of regex-like patterns that succinctly describe the syntactic variations in the strings. Real-world datasets, typically curated from multiple sources, often contain data in various syntactic formats. Thus, any data processing task is preceded by the critical step of data format identification. However, manual inspection of data to identify the different formats is infeasible in standard big-data scenarios. Prior techniques are restricted to a small set of pre-defined patterns (e.g. digits, letters, words, etc.), and provide no control over granularity of profiles. We define syntactic profiling as a problem of clustering strings based on syntactic similarity, followed by identifying patterns that succinctly describe each cluster. We present a technique for synthesizing such profiles over a given language of patterns, that also allows for interactive refinement by requesting a desired number of clusters. Using a state-of-the-art inductive synthesis framework, PROSE, we have implemented our technique as FlashProfile. Across 153 tasks over 75 large real datasets, we observe a median profiling time of only ~,0.7,s. Furthermore, we show that access to syntactic profiles may allow for more accurate synthesis of programs, i.e. using fewer examples, in programming-by-example (PBE) workflows such as FlashFill.