# LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs

arXiv: https://arxiv.org/abs/2604.08752 · HTML: https://mlsanity.com/paper/2604.08752/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 16 of 16)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [Pointer Graph Networks](https://mlsanity.com/paper/2006.06380/) (2020)
- [Simpler but More Accurate Semantic Dependency Parsing](https://mlsanity.com/paper/1807.01396/) (2018)
- [Autoregressive Structured Prediction with Language Models](https://mlsanity.com/paper/2210.14698/) (2022)
- [Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction](https://mlsanity.com/paper/1802.10569/) (2018)
- [SciDTB: Discourse Dependency TreeBank for Scientific Abstracts](https://mlsanity.com/paper/1806.03653/) (2018)
- [Understanding and Improving Information Preservation in Prompt Compression for LLMs](https://mlsanity.com/paper/2503.19114/) (2025)
- [UniRel: Unified Representation and Interaction for Joint Relational Triple Extraction](https://mlsanity.com/paper/2211.09039/) (2022)
- [Latent Graph Inference with Limited Supervision](https://mlsanity.com/paper/2310.04314/) (2023)
- [An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2401.01326/) (2024)

## Abstract (arXiv)

Relation extraction represents a fundamental component in the process of creating knowledge graphs, among other applications. Large language models (LLMs) have been adopted as a promising tool for relation extraction, both in supervised and in-context learning settings. However, in this work we show that their performance still lags behind much smaller architectures when the linguistic graph underlying a text has great complexity. To demonstrate this, we evaluate four LLMs against a graph-based parser on six relation extraction datasets with sentence graphs of varying sizes and complexities. Our results show that the graph-based parser increasingly outperforms the LLMs, as the number of relations in the input documents increases. This makes the much lighter graph-based parser a superior choice in the presence of complex linguistic graphs.