# Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysis

arXiv: https://arxiv.org/abs/2402.11398 · HTML: https://mlsanity.com/paper/2402.11398/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 17 Feb 2024.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [Improving Palliative Care with Deep Learning](https://mlsanity.com/paper/1711.06402/) (2017)
- [MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering](https://mlsanity.com/paper/2309.16035/) (2023)
- [Holistic Evaluation of GPT-4V for Biomedical Imaging](https://mlsanity.com/paper/2312.05256/) (2023)
- [RadOnc-GPT: A Large Language Model for Radiation Oncology](https://mlsanity.com/paper/2309.10160/) (2023)
- [ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data](https://mlsanity.com/paper/2310.05242/) (2023)
- [MediViSTA: Medical Video Segmentation via Temporal Fusion SAM Adaptation for Echocardiography](https://mlsanity.com/paper/2309.13539/) (2023)
- [Ophtha-LLaMA2: A Large Language Model for Ophthalmology](https://mlsanity.com/paper/2312.04906/) (2023)
- [Evaluating the Potential of Leading Large Language Models in Reasoning Biology Questions](https://mlsanity.com/paper/2311.07582/) (2023)

## Built on by (most cited first, 21 of 21)

- [A survey on fairness of large language models in e-commerce: progress, application, and challenge](https://mlsanity.com/paper/2405.13025/) (2024)
- [SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations](https://mlsanity.com/paper/2510.04398/) (2025)
- [Position: Towards a Responsible LLM-empowered Multi-Agent Systems](https://mlsanity.com/paper/2502.01714/) (2025)
- [Survey: Multi-Armed Bandits Meet Large Language Models](https://mlsanity.com/paper/2505.13355/) (2025)
- [Stop Reducing Responsibility in LLM-Powered Multi-Agent Systems to Local Alignment](https://mlsanity.com/paper/2510.14008/) (2025)
- [Palisade -- Prompt Injection Detection Framework](https://mlsanity.com/paper/2410.21146/) (2024)
- [TelcoAgent-Bench: A Multilingual Benchmark for Telecom AI Agents](https://mlsanity.com/paper/2604.06209/) (2026)
- [Large Language Models for Bioinformatics](https://mlsanity.com/paper/2501.06271/) (2025)
- [CoordField: Coordination Field for Agentic UAV Task Allocation In Low-altitude Urban Scenarios](https://mlsanity.com/paper/2505.00091/) (2025)
- [Opportunities and challenges in the application of large artificial intelligence models in radiology](https://mlsanity.com/paper/2403.16112/) (2024)
- [Transit Pulse: Utilizing Social Media as a Source for Customer Feedback and Information Extraction with Large Language Model](https://mlsanity.com/paper/2410.15016/) (2024)
- [Ensembling Multilingual Transformers for Robust Sentiment Analysis of Tweets](https://mlsanity.com/paper/2509.24080/) (2025)
- [Advances and Challenges in Semantic Textual Similarity: A Comprehensive Survey](https://mlsanity.com/paper/2601.03270/) (2025)
- [Evaluating LLMs for Police Decision-Making: A Framework Based on Police Action Scenarios](https://mlsanity.com/paper/2601.03553/) (2026)
- [Bridging Context Gaps: Enhancing Comprehension in Long-Form Social Conversations Through Contextualized Excerpts](https://mlsanity.com/paper/2412.19966/) (2024)
- [Language modelling techniques for analysing the impact of human genetic variation](https://mlsanity.com/paper/2503.10655/) (2025)
- [From Divergence to Consensus: Evaluating the Role of Large Language Models in Facilitating Agreement through Adaptive Strategies](https://mlsanity.com/paper/2503.15521/) (2025)
- [Semantic Similarity in Radiology Reports via LLMs and NER](https://mlsanity.com/paper/2510.03102/) (2025)
- [Real-Time Health Analytics Using Ontology-Driven Complex Event Processing and LLM Reasoning: A Tuberculosis Case Study](https://mlsanity.com/paper/2510.09646/) (2025)
- [Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relations](https://mlsanity.com/paper/2512.00556/) (2025)
- [A Dual-Dimensional LLM Framework for Automated Item Incidental Content Similarity Analysis in Large-Scale Assessments](https://mlsanity.com/paper/2608.24825/) (2026)

## Abstract (arXiv)

In this study, we leverage LLM to enhance the semantic analysis and develop similarity metrics for texts, addressing the limitations of traditional unsupervised NLP metrics like ROUGE and BLEU. We develop a framework where LLMs such as GPT-4 are employed for zero-shot text identification and label generation for radiology reports, where the labels are then used as measurements for text similarity. By testing the proposed framework on the MIMIC data, we find that GPT-4 generated labels can significantly improve the semantic similarity assessment, with scores more closely aligned with clinical ground truth than traditional NLP metrics. Our work demonstrates the possibility of conducting semantic analysis of the text data using semi-quantitative reasoning results by the LLMs for highly specialized domains. While the framework is implemented for radiology report similarity analysis, its concept can be extended to other specialized domains as well.