# In-Domain Supervised Pathology Report Classification: A Reproducible Pipeline from Data Curation to Production-Matched Evaluation

arXiv: https://arxiv.org/abs/2606.16026 · HTML: https://mlsanity.com/paper/2606.16026/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 13 references that are in the map.
- First posted on arXiv 14 Jun 2026.

## Builds on (most cited first, 1 of 1)

- [Hierarchical Deep Learning Classification of Unstructured Pathology Reports to Automate ICD-O Morphology Grading](https://mlsanity.com/paper/2009.00542/) (2020)

## Abstract (arXiv)

We introduce an in-domain supervised pipeline designed to counter the out-of-distribution performance drop that hampers supervised biomedical NLP models, a problem observed when models trained on pathology reports are moved across cancer registries. Our contribution is a reproducible recipe for training a supervised classifier from routinely collected cancer registry data. It describes how to build the in-domain training set and a production-matched holdout, and to choose operating points that keep the false-negative rate (FNR) very low while keeping reviewer workload manageable. The pipeline standardizes data curation with facility-stratified sampling and separate handling of reports linked to registry cases, and includes a blinded manual audit to estimate positive-case prevalence and label noise. On a 418k-report holdout set, the Kentucky model achieved FNR 0.003 and false-positive rate (FPR) 0.097, improving over the Seattle-trained MOSSAIC OncoID baseline (FNR 0.010, FPR 0.183) and raising F1 from 0.860 to 0.922. In a blinded manual review of 600 reports, estimated positive prevalence declined from 0.500 to 0.398, indicating substantial label noise with errors concentrated in rare primary sites.