# Explainable identification of similarities between entities for discovery in large text

arXiv: https://arxiv.org/abs/2503.17605 · HTML: https://mlsanity.com/paper/2503.17605/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 50 references that are in the map.
- First posted on arXiv 22 Mar 2025.

## Builds on (most cited first, 6 of 6)

- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Text Classification Algorithms: A Survey](https://mlsanity.com/paper/1904.08067/) (2019)
- [Deep Learning Based Text Classification: A Comprehensive Review](https://mlsanity.com/paper/2004.03705/) (2020)
- [Multi-document Summarization via Deep Learning Techniques: A Survey](https://mlsanity.com/paper/2011.04843/) (2020)
- [Legal Case Document Similarity: You Need Both Network and Text](https://mlsanity.com/paper/2209.12474/) (2022)
- [A data science and machine learning approach to continuous analysis of Shakespeare's plays](https://mlsanity.com/paper/2301.06024/) (2023)

## Built on by (most cited first, 1 of 1)

- [AI-Driven Cybersecurity Threats: A Survey of Emerging Risks and Defensive Strategies](https://mlsanity.com/paper/2601.03304/) (2026)

## Abstract (arXiv)

With the availability of virtually infinite number text documents in digital format, automatic comparison of textual data is essential for extracting meaningful insights that are difficult to identify manually. Many existing tools, including AI and large language models, struggle to provide precise and explainable insights into textual similarities. In many cases they determine the similarity between documents as reflected by the text, rather than the similarities between the subjects being discussed in these documents. This study addresses these limitations by developing an n-gram analysis framework designed to compare documents automatically and uncover explainable similarities. A scoring formula is applied to assigns each of the n-grams with a weight, where the weight is higher when the n-grams are more frequent in both documents, but is penalized when the n-grams are more frequent in the English language. Visualization tools like word clouds enhance the representation of these patterns, providing clearer insights. The findings demonstrate that this framework effectively uncovers similarities between text documents, offering explainable insights that are often difficult to identify manually. This non-parametric approach provides a deterministic solution for identifying similarities across various fields, including biographies, scientific literature, historical texts, and more. Code for the method is publicly available.