NLPCC 2026 Task 10: Citation-Level Faithfulness Verification with DeBERTa Ensembles and Class-Wise Calibration
Not on the map. Why?
Abstract (arXiv)
This paper presents our system for Track 2 of the NLPCC 2026 Shared Task 10 on citation-level faithfulness in AI-assisted scientific reporting. Given an atomic scientific claim and the structured full text of its cited paper, the task requires both a four-way relation label and up to three evidence paragraph identifiers. The label head ensembles a paragraph-aware cross-encoder with a document-level DeBERTa-large classifier, followed by class-wise decision calibration. Probability-level fusion is motivated by an out-of-fold tendency to over-predict Topical Match. The evidence head combines paragraph scores from top-20 and top-30 joint models with BM25 scores. The system runs fully offline without external retrieval or LLM prompting. On the final leaderboard, our system achieved 82.9898 overall (89.5491 Macro-F1 and 76.4305 Joint@3), ranking second in Track 2. Ablations and error analysis show that model complementarity and calibration drive the label gains. Gold-evidence inference changes label Macro-F1 negligibly, whereas evidence ranking remains important for Joint@3.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 10 references that are in the map. First posted on arXiv 19 Sep 2026.
Builds on
6 of 10 references.
References · 6
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
- 2020Fact or Fiction: Verifying Scientific Claimscited by 554
- 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
Built on by
Not cited yet.