# CACER: Clinical Concept Annotations for Cancer Events and Relations

arXiv: https://arxiv.org/abs/2409.03905 · HTML: https://mlsanity.com/paper/2409.03905/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 75 references that are in the map.
- First posted on arXiv 5 Sep 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains](https://mlsanity.com/paper/2402.10373/) (2024)
- [Span-based Joint Entity and Relation Extraction with Transformer Pre-training](https://mlsanity.com/paper/1909.07755/) (2019)
- [ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation](https://mlsanity.com/paper/2306.02022/) (2023)
- [Self-Verification Improves Few-Shot Clinical Information Extraction](https://mlsanity.com/paper/2306.00024/) (2023)
- [Packed Levitated Marker for Entity and Relation Extraction](https://mlsanity.com/paper/2109.06067/) (2021)
- [BERT-based Ranking for Biomedical Entity Normalization](https://mlsanity.com/paper/1908.03548/) (2019)
- [The 2022 n2c2/UW Shared Task on Extracting Social Determinants of Health](https://mlsanity.com/paper/2301.05571/) (2023)
- [Zero-shot information extraction from radiological reports using ChatGPT](https://mlsanity.com/paper/2309.01398/) (2023)
- [LeafAI: query generator for clinical cohort discovery rivaling a human programmer](https://mlsanity.com/paper/2304.06203/) (2023)
- [Relation extraction from clinical texts using domain invariant convolutional neural network](https://mlsanity.com/paper/1606.09370/) (2016)
- [Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning](https://mlsanity.com/paper/2306.07170/) (2023)
- [Leveraging Natural Language Processing to Augment Structured Social Determinants of Health Data in the Electronic Health Record](https://mlsanity.com/paper/2212.07538/) (2022)
- [A frame semantic overview of NLP-based information extraction for cancer-related EHR notes](https://mlsanity.com/paper/1904.01655/) (2019)
- [Extracting Social Determinants of Health from Pediatric Patient Notes Using Large Language Models: Novel Corpus and Methods](https://mlsanity.com/paper/2404.00826/) (2024)
- [A Novel Corpus of Annotated Medical Imaging Reports and Information Extraction Results Using BERT-based Language Models](https://mlsanity.com/paper/2403.18975/) (2024)
- [Generalizing through Forgetting -- Domain Generalization for Symptom Event Extraction in Clinical Notes](https://mlsanity.com/paper/2209.09485/) (2022)

## Built on by (most cited first, 2 of 2)

- [Medalyze: Lightweight Medical Report Summarization Application Using FLAN-T5-Large](https://mlsanity.com/paper/2505.17059/) (2025)
- [Ontology-Based Post-Coordinated Expression Assembly: from Clinical Narrative to Queryable Knowledge Graphs](https://mlsanity.com/paper/2610.16474/) (2026)

## Abstract (arXiv)

Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. We present Clinical Concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48,000 medical problems and drug events and 10,000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction (IE) models such as BERT, Flan-T5, Llama3, and GPT-4 using fine-tuning and in-context learning (ICL). In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, LLMs offer no performance advantage over the smaller BERT models. The results emphasize the need for annotated training data to optimize models. Multiple fine-tuned transformer models achieved performance comparable to IAA for several extraction tasks.