Clinical Intent Extraction: A FHIR-Aligned Representation and the CIRCA Benchmark
Not on the map. Why?
Abstract (arXiv)
Prospective clinical actions, the follow-ups, orders, referrals, and instructions that deter-mine what happens to a patient next, are annotated today in thin fragments across incom-patible corpora: each records a text span and one coarse category. We introduce Clinical Intent Extraction (CIE), the task of recovering these actions as complete structured rec-ords, and the Clinical Intent Representation (CIR), which decomposes each action into its verb, type, coded target, timing, and condition, and adds two axes prior datasets do not jointly represent: request-intent, the authority behind the action (proposal, plan, order, or option, aligned to HL7 FHIR), and modality, a seven-valued scale of clinical strength. Re-expressing five heterogeneous corpora (CLIP, MedDec, ap_parsing, PaniniQA, SIMORD) in the CIR yields CIRCA: 10,011 harmonized intents spanning two note distributions, with a human-validated subset, source-to-CIR crosswalks, and a deterministic FHIR R4 mapper. CIRCA is built by three-model consensus that auto-accepts high-agreement in-tents and routes the rest to human review; the audited agreement stratum matches human decisions 88.4% of the time. Benchmarking five existing models without task-specific training exposes the gap CIRCA targets: given the span, they label type well (85 to 91%) but get all four closed fields right only 18 to 35% of the time. All artifacts are released, with MIMIC-derived layers shared as stand-off annotations under PhysioNet credentialed access.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 45 references that are in the map. First posted on arXiv 24 Aug 2026.
Builds on
23 of 45 references.
References · 23
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
- 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
- 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
- 2023Can Large Language Models Transform Computational Social Science?cited by 245
- 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
- 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 151
- 2023ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generationcited by 116
- 2023ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learningcited by 97
- 2023LLMs Accelerate Annotation for Medical Information Extractioncited by 77
11 more
- 2023Automated Annotation with Generative AI Requires Validationcited by 75
- 2023Publicly Shareable Clinical Large Language Model Built on Synthetic Clinical Notescited by 51
- 2023PaniniQA: Enhancing Patient Education Through Interactive Question Answeringcited by 21
- 2019Towards Automatic Generation of Shareable Synthetic Clinical Notes Using Neural Language Modelscited by 20
- 2021CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notescited by 18
- 2019Extraction and Analysis of Clinically Important Follow-up Recommendations in a Large Radiology Datasetcited by 5
- 2025Overview of the MEDIQA-OE 2025 Shared Task on Medical Order Extraction from Doctor-Patient Consultationscited by 2
- 2026IRC-Bench: Recognizing Entities from Contextual Cues in First-Person Reminiscencescited by 2
- 2026CoEval: Ranking Language Models for Custom Tasks Without Labeled Data or Trustworthy Benchmarkscited by 2
- 2022Predicting Clinical Intent from Free Text Electronic Health Recordscited by 1
- 2026Three Generations of Healthcare IT: From the Digital Record to the Computable Care Processcited by 1
Built on by
Not cited yet.