# Cross-Domain Contract Element Extraction with a Bi-directional Feedback Clause-Element Relation Network

arXiv: https://arxiv.org/abs/2105.06083 · HTML: https://mlsanity.com/paper/2105.06083/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [Graph Adversarial Training: Dynamically Regularizing Based on Graph Structure](https://mlsanity.com/paper/1902.08226/) (2019)
- [Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks](https://mlsanity.com/paper/1703.06345/) (2017)
- [NCRF++: An Open-source Neural Sequence Labeling Toolkit](https://mlsanity.com/paper/1806.05626/) (2018)
- [Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method](https://mlsanity.com/paper/2010.11797/) (2020)
- [Building Legal Case Retrieval Systems with Lexical Matching and Summarization using A Pre-Trained Phrase Scoring Model](https://mlsanity.com/paper/2009.14083/) (2020)
- [Transfer Learning for Named-Entity Recognition with Neural Networks](https://mlsanity.com/paper/1705.06273/) (2017)
- [Neural Adaptation Layers for Cross-domain Named Entity Recognition](https://mlsanity.com/paper/1810.06368/) (2018)
- [Legal Document Retrieval using Document Vector Embeddings and Deep Learning](https://mlsanity.com/paper/1805.10685/) (2018)
- [Legal Question Answering using Ranking SVM and Deep Convolutional Neural Network](https://mlsanity.com/paper/1703.05320/) (2017)
- [Effective Use of Bidirectional Language Modeling for Transfer Learning in Biomedical Named Entity Recognition](https://mlsanity.com/paper/1711.07908/) (2017)
- [Neural Contract Element Extraction Revisited: Letters from Sesame Street](https://mlsanity.com/paper/2101.04355/) (2021)
- [Named Entity Recognition for Novel Types by Transfer Learning](https://mlsanity.com/paper/1610.09914/) (2016)

## Built on by (most cited first, 3 of 3)

- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [ConReader: Exploring Implicit Relations in Contracts for Contract Clause Extraction](https://mlsanity.com/paper/2210.08697/) (2022)
- [RealKIE: Five Novel Datasets for Enterprise Key Information Extraction](https://mlsanity.com/paper/2403.20101/) (2024)

## Abstract (arXiv)

Contract element extraction (CEE) is the novel task of automatically identifying and extracting legally relevant elements such as contract dates, payments, and legislation references from contracts. Automatic methods for this task view it as a sequence labeling problem and dramatically reduce human labor. However, as contract genres and element types may vary widely, a significant challenge for this sequence labeling task is how to transfer knowledge from one domain to another, i.e., cross-domain CEE. Cross-domain CEE differs from cross-domain named entity recognition (NER) in two important ways. First, contract elements are far more fine-grained than named entities, which hinders the transfer of extractors. Second, the extraction zones for cross-domain CEE are much larger than for cross-domain NER. As a result, the contexts of elements from different domains can be more diverse. We propose a framework, the Bi-directional Feedback cLause-Element relaTion network (Bi-FLEET), for the cross-domain CEE task that addresses the above challenges. Bi-FLEET has three main components: (1) a context encoder, (2) a clause-element relation encoder, and (3) an inference layer. To incorporate invariant knowledge about element and clause types, a clause-element graph is constructed across domains and a hierarchical graph neural network is adopted in the clause-element relation encoder. To reduce the influence of context variations, a multi-task framework with a bi-directional feedback scheme is designed in the inference layer, conducting both clause classification and element extraction. The experimental results over both cross-domain NER and CEE tasks show that Bi-FLEET significantly outperforms state-of-the-art baselines.