# Semantic Reranking at Inference Time for Hard Examples in Rhetorical Role Labeling

arXiv: https://arxiv.org/abs/2605.18007 · HTML: https://mlsanity.com/paper/2605.18007/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 18 May 2026.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [SaulLM-7B: A pioneering Large Language Model for Law](https://mlsanity.com/paper/2403.03883/) (2024)
- [PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts](https://mlsanity.com/paper/1710.06071/) (2017)
- [Uncertainty in Language Models: Assessment through Rank-Calibration](https://mlsanity.com/paper/2404.03163/) (2024)
- [MAQA: Evaluating Uncertainty Quantification in LLMs Regarding Data Uncertainty](https://mlsanity.com/paper/2408.06816/) (2024)
- [Corpus for Automatic Structuring of Legal Documents](https://mlsanity.com/paper/2201.13125/) (2022)
- [When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs](https://mlsanity.com/paper/2508.02994/) (2025)
- [HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification](https://mlsanity.com/paper/2403.17307/) (2024)
- [Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-following LLM](https://mlsanity.com/paper/2304.11872/) (2023)
- [Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Papers](https://mlsanity.com/paper/2102.06008/) (2021)
- [Harnessing GPT-3.5-turbo for Rhetorical Role Prediction in Legal Cases](https://mlsanity.com/paper/2310.17413/) (2023)
- [Incubating Text Classifiers Following User Instruction with Nothing but LLM](https://mlsanity.com/paper/2404.10877/) (2024)
- [HiCuLR: Hierarchical Curriculum Learning for Rhetorical Role Labeling of Legal Documents](https://mlsanity.com/paper/2409.18647/) (2024)
- [Dynamic Label Name Refinement for Few-Shot Dialogue Intent Classification](https://mlsanity.com/paper/2412.15603/) (2024)
- [Mind Your Neighbours: Leveraging Analogous Instances for Rhetorical Role Labeling for Legal Documents](https://mlsanity.com/paper/2404.01344/) (2024)
- [Evaluating Design Decisions for Dual Encoder-based Entity Disambiguation](https://mlsanity.com/paper/2505.11683/) (2025)
- [Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches](https://mlsanity.com/paper/2512.12677/) (2025)
- [Joint Span Segmentation and Rhetorical Role Labeling with Data Augmentation for Legal Documents](https://mlsanity.com/paper/2302.06448/) (2023)
- [Label-semantics Aware Generative Approach for Domain-Agnostic Multilabel Classification](https://mlsanity.com/paper/2506.06806/) (2025)

## Abstract (arXiv)

Rhetorical Role Labeling (RRL) assigns a functional role to each sentence in a document and is widely used in legal, medical, and scientific domains. While language models (LMs) achieve strong average performance, they remain unreliable on hard examples, where prediction confidence is low. Existing approaches typically handle uncertainty implicitly and treat labels as discrete identifiers, overlooking the semantic information encoded in label names. We introduce RISE, an inference-time semantic reranking framework that leverages label semantics to refine predictions on hard instances. RISE automatically identifies low-confidence predictions and reranks model outputs using contrastively learned label representations, without retraining or modifying the underlying model. Experiments on eight domain-specific RRL datasets with seven LMs, including encoder-based and causal architectures, show an average gain of +9.15 macro-F1 points on hard examples. For explainability, we further propose manual hardness annotations to study difficulty from both model and human perspectives, revealing a moderate agreement with Cohen's kappa = 0.40.