mlsanityData to 14 Oct 2026

TocBERT: Medical Document Structure Extraction Using Bidirectional Transformers

arXiv 2406.19526 ↗First posted 27 Jun 2024cs.CL cs.IR cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Text segmentation holds paramount importance in the field of Natural Language Processing (NLP). It plays an important role in several NLP downstream tasks like information retrieval and document summarization. In this work, we propose a new solution, namely TocBERT, for segmenting texts using bidirectional transformers. TocBERT represents a supervised solution trained on the detection of titles and sub-titles from their semantic representations. This task was formulated as a named entity recognition (NER) problem. The solution has been applied on a medical text segmentation use-case where the Bio-ClinicalBERT model is fine-tuned to segment discharge summaries of the MIMIC-III dataset. The performance of TocBERT has been evaluated on a human-labeled ground truth corpus of 250 notes. It achieved an F1-score of 84.6% when evaluated on a linear text segmentation problem and 72.8% on a hierarchical text segmentation problem. It outperformed a carefully designed rule-based solution, particularly in distinguishing titles from subtitles.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 19 of its 32 references that are in the map. First posted on arXiv 27 Jun 2024.

Builds on

19 of 32 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  8. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  9. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  10. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  11. 2019PubLayNet: largest dataset ever for document layout analysiscited by 351
  12. 2023Large Language Models for Information Retrieval: A Surveycited by 331
7 more
  1. 2018Text Segmentation as a Supervised Learning Taskcited by 76
  2. 2020Text Segmentation by Cross Segment Attentioncited by 54
  3. 2020Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentationcited by 41
  4. 2021Unsupervised Topic Segmentation of Meetings with BERT Embeddingscited by 30
  5. 2015Text Segmentation based on Semantic Word Embeddingscited by 11
  6. 2023Document Summarization with Text Segmentationcited by 3
  7. 2024Anatomy of Neural Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Bridging the Domain Divide: Supervised vs. Zero-Shot Clinical Section Segmentation from MIMIC-III to Obstetricscited by 2
  2. 2025Clinical Document Metadata Extraction: A Scoping Reviewcited by 0
  3. 2026Towards Multidisciplinary Summarization of Hospital Stays: Efficient Sentence-Level Clinical Provenance Categorizationcited by 0