# An Auditable Pipeline for Fuzzy Full-Text Screening in Systematic Reviews: Integrating Contrastive Semantic Highlighting and LLM Judgment

arXiv: https://arxiv.org/abs/2508.15822 · HTML: https://mlsanity.com/paper/2508.15822/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 37 references that are in the map.
- First posted on arXiv 17 Aug 2025.

## Builds on (most cited first, 8 of 8)

- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [Automated Paper Screening for Clinical Reviews Using Large Language Models](https://mlsanity.com/paper/2305.00844/) (2023)
- [Text Clustering with Large Language Model Embeddings](https://mlsanity.com/paper/2403.15112/) (2024)
- [The emergence of Large Language Models (LLM) as a tool in literature reviews: an LLM automated systematic review](https://mlsanity.com/paper/2409.04600/) (2024)
- [Exploring the Use of ChatGPT as a Tool for Learning and Assessment in Undergraduate Computer Science Curriculum: Opportunities and Challenges](https://mlsanity.com/paper/2304.11214/) (2023)
- [FAME: Introducing Fuzzy Additive Models for Explainable AI](https://mlsanity.com/paper/2504.07011/) (2025)
- [Text Chunking for Document Classification for Urban System Management using Large Language Models](https://mlsanity.com/paper/2504.00274/) (2025)

## Abstract (arXiv)

Full-text screening is the major bottleneck of systematic reviews (SRs), as decisive evidence is dispersed across long, heterogeneous documents and rarely admits static, binary rules. We present a scalable, auditable pipeline that reframes inclusion/exclusion as a fuzzy decision problem and benchmark it against statistical and crisp baselines in the context of the Population Health Modelling Consensus Reporting Network for noncommunicable diseases (POPCORN). Articles are parsed into overlapping chunks and embedded with a domain-adapted model; for each criterion (Population, Intervention, Outcome, Study Approach), we compute contrastive similarity (inclusion-exclusion cosine) and a vagueness margin, which a Mamdani fuzzy controller maps into graded inclusion degrees with dynamic thresholds in a multi-label setting. A large language model (LLM) judge adjudicates highlighted spans with tertiary labels, confidence scores, and criterion-referenced rationales; when evidence is insufficient, fuzzy membership is attenuated rather than excluded. In a pilot on an all-positive gold set (16 full texts; 3,208 chunks), the fuzzy system achieved recall of 81.3% (Population), 87.5% (Intervention), 87.5% (Outcome), and 75.0% (Study Approach), surpassing statistical (56.3-75.0%) and crisp baselines (43.8-81.3%). Strict "all-criteria" inclusion was reached for 50.0% of articles, compared to 25.0% and 12.5% under the baselines. Cross-model agreement on justifications was 98.3%, human-machine agreement 96.1%, and a pilot review showed 91% inter-rater agreement (kappa = 0.82), with screening time reduced from about 20 minutes to under 1 minute per article at significantly lower cost. These results show that fuzzy logic with contrastive highlighting and LLM adjudication yields high recall, stable rationale, and end-to-end traceability.