# Is There a One-Model-Fits-All Approach to Information Extraction? Revisiting Task Definition Biases

arXiv: https://arxiv.org/abs/2403.16396 · HTML: https://mlsanity.com/paper/2403.16396/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 22 of 22)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://mlsanity.com/paper/2309.11235/) (2023)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction](https://mlsanity.com/paper/2310.03668/) (2023)
- [CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors](https://mlsanity.com/paper/2305.05711/) (2023)
- [A Hierarchical Framework for Relation Extraction with Reinforcement Learning](https://mlsanity.com/paper/1811.03925/) (2018)
- [Universal Information Extraction as Unified Semantic Matching](https://mlsanity.com/paper/2301.03282/) (2023)
- [Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors](https://mlsanity.com/paper/2305.11159/) (2023)
- [Retrieval-Augmented Code Generation for Universal Information Extraction](https://mlsanity.com/paper/2311.02962/) (2023)
- [Improving Distantly Supervised Relation Extraction using Word and Entity Based Attention](https://mlsanity.com/paper/1804.06987/) (2018)
- [POLYGLOT-NER: Massive Multilingual Named Entity Recognition](https://mlsanity.com/paper/1410.3791/) (2014)
- [Named Entity Recognition in Twitter: A Dataset and Analysis on Short-Term Temporal Shifts](https://mlsanity.com/paper/2210.03797/) (2022)
- [Learning to Transfer Prompts for Text Generation](https://mlsanity.com/paper/2205.01543/) (2022)
- [Revisiting the Negative Data of Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2105.10158/) (2021)

## Built on by (most cited first, 1 of 1)

- [SelectLLM: Query-Aware Efficient Selection Algorithm for Large Language Models](https://mlsanity.com/paper/2408.08545/) (2024)

## Abstract (arXiv)

Definition bias is a negative phenomenon that can mislead models. Definition bias in information extraction appears not only across datasets from different domains but also within datasets sharing the same domain. We identify two types of definition bias in IE: bias among information extraction datasets and bias between information extraction datasets and instruction tuning datasets. To systematically investigate definition bias, we conduct three probing experiments to quantitatively analyze it and discover the limitations of unified information extraction and large language models in solving definition bias. To mitigate definition bias in information extraction, we propose a multi-stage framework consisting of definition bias measurement, bias-aware fine-tuning, and task-specific bias mitigation. Experimental results demonstrate the effectiveness of our framework in addressing definition bias. Resources of this paper can be found at https://github.com/EZ-hwh/definition-bias