mlsanityData to 7 Oct 2026

Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysis

arXiv 2402.11398 ↗First posted 17 Feb 2024cs.CL cs.AIcited by 21

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

In this study, we leverage LLM to enhance the semantic analysis and develop similarity metrics for texts, addressing the limitations of traditional unsupervised NLP metrics like ROUGE and BLEU. We develop a framework where LLMs such as GPT-4 are employed for zero-shot text identification and label generation for radiology reports, where the labels are then used as measurements for text similarity. By testing the proposed framework on the MIMIC data, we find that GPT-4 generated labels can significantly improve the semantic similarity assessment, with scores more closely aligned with clinical ground truth than traditional NLP metrics. Our work demonstrates the possibility of conducting semantic analysis of the text data using semi-quantitative reasoning results by the LLMs for highly specialized domains. While the framework is implemented for radiology report similarity analysis, its concept can be extended to other specialized domains as well.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 13 of its 32 references that are in the map. First posted on arXiv 17 Feb 2024.

Builds on

13 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Evaluating the Potential of Leading Large Language Models in Reasoning Biology Questions5 Nov

Other references · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  4. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  5. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  6. 2017Improving Palliative Care with Deep Learningcited by 28
  7. 2023MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answeringcited by 23
  8. 2023Holistic Evaluation of GPT-4V for Biomedical Imagingcited by 19
  9. 2023RadOnc-GPT: A Large Language Model for Radiation Oncologycited by 18
  10. 2023ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Datacited by 14
  11. 2023MediViSTA: Medical Video Segmentation via Temporal Fusion SAM Adaptation for Echocardiographycited by 10
  12. 2023Ophtha-LLaMA2: A Large Language Model for Ophthalmologycited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 13; 2026: 3.

21 papers.

Sort:

By area:

  1. 2024A survey on fairness of large language models in e-commerce: progress, application, and challengecited by 10
  2. 2025SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinationscited by 9
  3. 2025Position: Towards a Responsible LLM-empowered Multi-Agent Systemscited by 7
  4. 2025Stop Reducing Responsibility in LLM-Powered Multi-Agent Systems to Local Alignmentcited by 7
  5. 2025Survey: Multi-Armed Bandits Meet Large Language Modelscited by 6
  6. 2026TelcoAgent-Bench: A Multilingual Benchmark for Telecom AI Agentscited by 4
  7. 2024Palisade -- Prompt Injection Detection Frameworkcited by 3
  8. 2025Large Language Models for Bioinformaticscited by 3
  9. 2025CoordField: Coordination Field for Agentic UAV Task Allocation In Low-altitude Urban Scenarioscited by 2
  10. 2024Opportunities and challenges in the application of large artificial intelligence models in radiologycited by 1
  11. 2024Transit Pulse: Utilizing Social Media as a Source for Customer Feedback and Information Extraction with Large Language Modelcited by 1
  12. 2025Ensembling Multilingual Transformers for Robust Sentiment Analysis of Tweetscited by 1
9 more
  1. 2025Advances and Challenges in Semantic Textual Similarity: A Comprehensive Surveycited by 1
  2. 2026Evaluating LLMs for Police Decision-Making: A Framework Based on Police Action Scenarioscited by 1
  3. 2024Bridging Context Gaps: Enhancing Comprehension in Long-Form Social Conversations Through Contextualized Excerptscited by 0
  4. 2025Language modelling techniques for analysing the impact of human genetic variationcited by 0
  5. 2025From Divergence to Consensus: Evaluating the Role of Large Language Models in Facilitating Agreement through Adaptive Strategiescited by 0
  6. 2025Semantic Similarity in Radiology Reports via LLMs and NERcited by 0
  7. 2025Real-Time Health Analytics Using Ontology-Driven Complex Event Processing and LLM Reasoning: A Tuberculosis Case Studycited by 0
  8. 2025Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relationscited by 0
  9. 2026A Dual-Dimensional LLM Framework for Automated Item Incidental Content Similarity Analysis in Large-Scale Assessmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Ontology-Based Concept Distillation for Radiology Report Retrieval and Labeling
  2. 2024MRScore: Evaluating Radiology Report Generation with LLM-based Reward System
  3. 2026Clinical Reasoning Graphs: Structured Evaluation of LLM Diagnostic Reasoning Reveals Competence Without Consistency
  4. 2026LLM-Guided Diagnostic Evidence Alignment for Medical Vision-Language Pretraining under Limited Pairing
  5. 2024Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for Radiology Reports
  6. 2026Measuring What VLMs Don't Say: Validation Metrics Hide Clinical Terminology Erasure in Radiology Report Generation
  7. 2025Evaluating Causal Explanation in Medical Reports with LLM-Based and Human-Aligned Metrics
  8. 2025Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generation