# A Novel DeBERTa-based Model for Financial Question Answering Task

arXiv: https://arxiv.org/abs/2207.05875 · HTML: https://mlsanity.com/paper/2207.05875/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering](https://mlsanity.com/paper/2104.06378/) (2021)
- [Variational Reasoning for Question Answering with Knowledge Graph](https://mlsanity.com/paper/1709.04071/) (2017)
- [Hurdles to Progress in Long-form Question Answering](https://mlsanity.com/paper/2103.06332/) (2021)
- [SPARQA: Skeleton-based Semantic Parsing for Complex Questions over Knowledge Bases](https://mlsanity.com/paper/2003.13956/) (2020)
- [Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases](https://mlsanity.com/paper/1903.02188/) (2019)

## Built on by (most cited first, 5 of 5)

- [Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey](https://mlsanity.com/paper/2305.18703/) (2023)
- [APOLLO: An Optimized Training Approach for Long-form Numerical Reasoning](https://mlsanity.com/paper/2212.07249/) (2022)
- [Learning Sparsity for Effective and Efficient Music Performance Question Answering](https://mlsanity.com/paper/2506.01319/) (2025)
- [Program of Thoughts for Financial Reasoning: Leveraging Dynamic In-Context Examples and Generative Retrieval](https://mlsanity.com/paper/2510.13157/) (2025)
- [From Documents to Reasoning: A Validated Synthetic Data Pipeline and Semantic-Aware Fine-Tuning for Financial Numerical Reasoning](https://mlsanity.com/paper/2608.27919/) (2026)

## Abstract (arXiv)

As a rising star in the field of natural language processing, question answering systems (Q&A Systems) are widely used in all walks of life. Compared with other scenarios, the applicationin financial scenario has strong requirements in the traceability and interpretability of the Q&A systems. In addition, since the demand for artificial intelligence technology has gradually shifted from the initial computational intelligence to cognitive intelligence, this research mainly focuses on the financial numerical reasoning dataset - FinQA. In the shared task, the objective is to generate the reasoning program and the final answer according to the given financial report containing text and tables. We use the method based on DeBERTa pre-trained language model, with additional optimization methods including multi-model fusion, training set combination on this basis. We finally obtain an execution accuracy of 68.99 and a program accuracy of 64.53, ranking No. 4 in the 2022 FinQA Challenge.