# DSC-IITISM at FinCausal 2021: Combining POS tagging with Attention-based Contextual Representations for Identifying Causal Relationships in Financial Documents

arXiv: https://arxiv.org/abs/2111.00490 · HTML: https://mlsanity.com/paper/2111.00490/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 31 Oct 2021.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Financial Document Causality Detection Shared Task (FinCausal 2020)](https://mlsanity.com/paper/2012.02505/) (2020)
- [Stock Movement Prediction with Financial News using Contextualized Embedding from BERT](https://mlsanity.com/paper/2107.08721/) (2021)
- [Uses and Abuses of the Cross-Entropy Loss: Case Studies in Modern Deep Learning](https://mlsanity.com/paper/2011.05231/) (2020)
- [Creating Causal Embeddings for Question Answering with Minimal Supervision](https://mlsanity.com/paper/1609.08097/) (2016)
- [NLP in FinTech Applications: Past, Present and Future](https://mlsanity.com/paper/2005.01320/) (2020)

## Abstract (arXiv)

Causality detection draws plenty of attention in the field of Natural Language Processing and linguistics research. It has essential applications in information retrieval, event prediction, question answering, financial analysis, and market research. In this study, we explore several methods to identify and extract cause-effect pairs in financial documents using transformers. For this purpose, we propose an approach that combines POS tagging with the BIO scheme, which can be integrated with modern transformer models to address this challenge of identifying causality in a given text. Our best methodology achieves an F1-Score of 0.9551, and an Exact Match Score of 0.8777 on the blind test in the FinCausal-2021 Shared Task at the FinCausal 2021 Workshop.