# Similar Data Points Identification with LLM: A Human-in-the-loop Strategy Using Summarization and Hidden State Insights

arXiv: https://arxiv.org/abs/2404.04281 · HTML: https://mlsanity.com/paper/2404.04281/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 3 Apr 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Ask Me Anything: A simple strategy for prompting language models](https://mlsanity.com/paper/2210.02441/) (2022)
- [Transformer-based unsupervised patient representation learning based on medical claims for risk stratification and analysis](https://mlsanity.com/paper/2106.12658/) (2021)
- [Human-in-the-loop model explanation via verbatim boundary identification in generated neighborhoods](https://mlsanity.com/paper/2106.13093/) (2021)
- [Pre-training transformer-based framework on large-scale pediatric claims data for downstream population-specific tasks](https://mlsanity.com/paper/2106.13095/) (2021)
- [Distributed representation of patients and its use for medical cost prediction](https://mlsanity.com/paper/1909.07157/) (2019)

## Built on by (most cited first, 9 of 9)

- [A Survey on Human-Centric LLMs](https://mlsanity.com/paper/2411.14491/) (2024)
- [Trust-Oriented Adaptive Guardrails for Large Language Models](https://mlsanity.com/paper/2408.08959/) (2024)
- [Leveraging Large Language Models for Entity Matching](https://mlsanity.com/paper/2405.20624/) (2024)
- [Translating Expert Intuition into Quantifiable Features: Encode Investigator Domain Knowledge via LLM for Enhanced Predictive Analytics](https://mlsanity.com/paper/2405.08017/) (2024)
- [Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring](https://mlsanity.com/paper/2502.05242/) (2025)
- [Evaluating LLM-Contaminated Crowdsourcing Data Without Ground Truth](https://mlsanity.com/paper/2506.06991/) (2025)
- [A Customer Level Fraudulent Activity Detection Benchmark for Enhancing Machine Learning Model Research and Evaluation](https://mlsanity.com/paper/2404.14746/) (2024)
- [Contrastive MIM: A Contrastive Mutual Information Framework for Unified Generative and Discriminative Representation Learning](https://mlsanity.com/paper/2502.19642/) (2025)
- [From Automation to Collaboration: Human-in-the-Loop Methods for Safe and Trustworthy NLP](https://mlsanity.com/paper/2605.25226/) (2026)

## Abstract (arXiv)

This study introduces a simple yet effective method for identifying similar data points across non-free text domains, such as tabular and image data, using Large Language Models (LLMs). Our two-step approach involves data point summarization and hidden state extraction. Initially, data is condensed via summarization using an LLM, reducing complexity and highlighting essential information in sentences. Subsequently, the summarization sentences are fed through another LLM to extract hidden states, serving as compact, feature-rich representations. This approach leverages the advanced comprehension and generative capabilities of LLMs, offering a scalable and efficient strategy for similarity identification across diverse datasets. We demonstrate the effectiveness of our method in identifying similar data points on multiple datasets. Additionally, our approach enables non-technical domain experts, such as fraud investigators or marketing operators, to quickly identify similar data points tailored to specific scenarios, demonstrating its utility in practical applications. In general, our results open new avenues for leveraging LLMs in data analysis across various domains