# Uncovering Main Causalities for Long-tailed Information Extraction

arXiv: https://arxiv.org/abs/2109.05213 · HTML: https://mlsanity.com/paper/2109.05213/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 80 references that are in the map.
- First posted on arXiv 11 Sep 2021.

## Builds on (most cited first, 30 of 51)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language](https://mlsanity.com/paper/1912.08830/) (2019)
- [Unbiased Scene Graph Generation from Biased Training](https://mlsanity.com/paper/2002.11949/) (2020)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [Counterfactual VQA: A Cause-Effect Look at Language Bias](https://mlsanity.com/paper/2006.04315/) (2020)
- [Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect](https://mlsanity.com/paper/2009.12991/) (2020)
- [Causal Intervention for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2009.12547/) (2020)
- [Scan2Cap: Context-aware Dense Captioning in RGB-D Scans](https://mlsanity.com/paper/2012.02206/) (2020)
- [Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning](https://mlsanity.com/paper/2010.15875/) (2020)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015)
- [Graph Convolution over Pruned Dependency Trees Improves Relation Extraction](https://mlsanity.com/paper/1809.10185/) (2018)
- [Causal Intervention for Leveraging Popularity Bias in Recommendation](https://mlsanity.com/paper/2105.06067/) (2021)
- [Interventional Few-Shot Learning](https://mlsanity.com/paper/2009.13000/) (2020)
- [The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation](https://mlsanity.com/paper/2007.11978/) (2020)
- [Context-aware Biaffine Localizing Network for Temporal Sentence Grounding](https://mlsanity.com/paper/2103.11555/) (2021)
- [MAVEN: A Massive General Domain Event Detection Dataset](https://mlsanity.com/paper/2004.13590/) (2020)
- [Cross-Sentence N-ary Relation Extraction with Graph LSTMs](https://mlsanity.com/paper/1708.03743/) (2017)
- [Reasoning with Latent Structure Refinement for Document-Level Relation Extraction](https://mlsanity.com/paper/2005.06312/) (2020)
- [Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System](https://mlsanity.com/paper/2010.15363/) (2020)
- [Interventional Video Grounding with Dual Contrastive Learning](https://mlsanity.com/paper/2106.11013/) (2021)
- [SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events](https://mlsanity.com/paper/2103.15538/) (2021)
- [Deconfounded Video Moment Retrieval with Causal Intervention](https://mlsanity.com/paper/2106.01534/) (2021)
- [Document-level Relation Extraction as Semantic Segmentation](https://mlsanity.com/paper/2106.03618/) (2021)
- [Learning from Context or Names? An Empirical Study on Neural Relation Extraction](https://mlsanity.com/paper/2010.01923/) (2020)
- [Robustness to Spurious Correlations in Text Classification via Automatically Generated Counterfactuals](https://mlsanity.com/paper/2012.10040/) (2020)

## Built on by (most cited first, 21 of 21)

- [Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango](https://mlsanity.com/paper/2209.07686/) (2022)
- [Prompting Large Language Models for Counterfactual Generation: An Empirical Study](https://mlsanity.com/paper/2305.14791/) (2023)
- [Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis](https://mlsanity.com/paper/2205.03784/) (2022)
- [A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing](https://mlsanity.com/paper/2210.04675/) (2022)
- [A Causal View of Entity Bias in (Large) Language Models](https://mlsanity.com/paper/2305.14695/) (2023)
- [Joint Vision-Language Social Bias Removal for CLIP](https://mlsanity.com/paper/2411.12785/) (2024)
- [Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction](https://mlsanity.com/paper/2210.04992/) (2022)
- [Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference](https://mlsanity.com/paper/2306.10790/) (2023)
- [Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework](https://mlsanity.com/paper/2409.16146/) (2024)
- [Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition](https://mlsanity.com/paper/2210.03980/) (2022)
- [Adaptive Visual Scene Understanding: Incremental Scene Graph Generation](https://mlsanity.com/paper/2310.01636/) (2023)
- [Think Rationally about What You See: Continuous Rationale Extraction for Relation Extraction](https://mlsanity.com/paper/2305.03503/) (2023)
- [Towards Heterogeneous Long-tailed Learning: Benchmarking, Metrics, and Toolbox](https://mlsanity.com/paper/2307.08235/) (2023)
- [Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study](https://mlsanity.com/paper/2210.10678/) (2022)
- [How Fragile is Relation Extraction under Entity Replacements?](https://mlsanity.com/paper/2305.13551/) (2023)
- [A Unified Causal View of Instruction Tuning](https://mlsanity.com/paper/2402.06220/) (2024)
- [Balancing the Causal Effects in Class-Incremental Learning](https://mlsanity.com/paper/2402.10063/) (2024)
- [Towards Generalized Open Information Extraction](https://mlsanity.com/paper/2211.15987/) (2022)
- [ReMask: A Robust Information-Masking Approach for Domain Counterfactual Generation](https://mlsanity.com/paper/2305.02858/) (2023)
- [Inducing Causal Structure for Abstractive Text Summarization](https://mlsanity.com/paper/2308.12888/) (2023)
- [Exploring Description-Augmented Dataless Intent Classification](https://mlsanity.com/paper/2407.17862/) (2024)

## Abstract (arXiv)

Information Extraction (IE) aims to extract structural information from unstructured texts. In practice, long-tailed distributions caused by the selection bias of a dataset, may lead to incorrect correlations, also known as spurious correlations, between entities and labels in the conventional likelihood models. This motivates us to propose counterfactual IE (CFIE), a novel framework that aims to uncover the main causalities behind data in the view of causal inference. Specifically, 1) we first introduce a unified structural causal model (SCM) for various IE tasks, describing the relationships among variables; 2) with our SCM, we then generate counterfactuals based on an explicit language structure to better calculate the direct causal effect during the inference stage; 3) we further propose a novel debiasing approach to yield more robust predictions. Experiments on three IE tasks across five public datasets show the effectiveness of our CFIE model in mitigating the spurious correlation issues.