# KPI-BERT: A Joint Named Entity Recognition and Relation Extraction Model for Financial Reports

arXiv: https://arxiv.org/abs/2208.02140 · HTML: https://mlsanity.com/paper/2208.02140/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 3 Aug 2022.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Span-based Joint Entity and Relation Extraction with Transformer Pre-training](https://mlsanity.com/paper/1909.07755/) (2019)
- [BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision](https://mlsanity.com/paper/2006.15509/) (2020)
- [Double Graph Based Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2009.13752/) (2020)
- [Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders](https://mlsanity.com/paper/2010.03851/) (2020)
- [Automated Concatenation of Embeddings for Structured Prediction](https://mlsanity.com/paper/2010.05006/) (2020)
- [Document-level Relation Extraction as Semantic Segmentation](https://mlsanity.com/paper/2106.03618/) (2021)
- [Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning](https://mlsanity.com/paper/2105.03654/) (2021)
- [Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction](https://mlsanity.com/paper/2102.10249/) (2021)
- [A Trigger-Sense Memory Flow Framework for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2101.10213/) (2021)
- [End-to-end Named Entity Recognition and Relation Extraction using Pre-trained Language Models](https://mlsanity.com/paper/1912.13415/) (2019)
- [Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers](https://mlsanity.com/paper/2010.04362/) (2020)

## Built on by (most cited first, 19 of 19)

- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [KPI-EDGAR: A Novel Dataset and Accompanying Metric for Relation Extraction from Financial Documents](https://mlsanity.com/paper/2210.09163/) (2022)
- [Informed Named Entity Recognition Decoding for Generative Language Models](https://mlsanity.com/paper/2308.07791/) (2023)
- [Improving Zero-Shot Text Matching for Financial Auditing with Large Language Models](https://mlsanity.com/paper/2308.06111/) (2023)
- [sustain.AI: a Recommender System to analyze Sustainability Reports](https://mlsanity.com/paper/2305.08711/) (2023)
- [Improving Legal Judgement Prediction in Romanian with Long Text Encoders](https://mlsanity.com/paper/2402.19170/) (2024)
- [Towards automating Numerical Consistency Checks in Financial Reports](https://mlsanity.com/paper/2211.06112/) (2022)
- [Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs](https://mlsanity.com/paper/2305.16344/) (2023)
- [A Linguistic Investigation of Machine Learning based Contradiction Detection Models: An Empirical Analysis and Future Perspectives](https://mlsanity.com/paper/2210.10434/) (2022)
- [Zero-Shot Text Matching for Automated Auditing using Sentence Transformers](https://mlsanity.com/paper/2211.07716/) (2022)
- [Enhancing Language Models for Financial Relation Extraction with Named Entities and Part-of-Speech](https://mlsanity.com/paper/2405.06665/) (2024)
- [Sustainable Digitalization of Business with Multi-Agent RAG and LLM](https://mlsanity.com/paper/2502.15700/) (2025)
- [Finding Stakeholder-Material Information from 10-K Reports using Fine-Tuned BERT and LSTM Models](https://mlsanity.com/paper/2308.07522/) (2023)
- [Controlled Randomness Improves the Performance of Transformer Models](https://mlsanity.com/paper/2310.13526/) (2023)
- [Transformer-based Named Entity Recognition in Construction Supply Chain Risk Management in Australia](https://mlsanity.com/paper/2311.13755/) (2023)
- [CO-Fun: A German Dataset on Company Outsourcing in Fund Prospectuses for Named Entity Recognition and Relation Extraction](https://mlsanity.com/paper/2403.15322/) (2024)
- [Pointer-Guided Pre-Training: Infusing Large Language Models with Paragraph-Level Contextual Awareness](https://mlsanity.com/paper/2406.04156/) (2024)
- [Extract Information from Hybrid Long Documents Leveraging LLMs: A Framework and Dataset](https://mlsanity.com/paper/2412.20072/) (2024)

## Abstract (arXiv)

We present KPI-BERT, a system which employs novel methods of named entity recognition (NER) and relation extraction (RE) to extract and link key performance indicators (KPIs), e.g. "revenue" or "interest expenses", of companies from real-world German financial documents. Specifically, we introduce an end-to-end trainable architecture that is based on Bidirectional Encoder Representations from Transformers (BERT) combining a recurrent neural network (RNN) with conditional label masking to sequentially tag entities before it classifies their relations. Our model also introduces a learnable RNN-based pooling mechanism and incorporates domain expert knowledge by explicitly filtering impossible relations. We achieve a substantially higher prediction performance on a new practical dataset of German financial reports, outperforming several strong baselines including a competing state-of-the-art span-based entity tagging approach.