# Model-Agnostic Meta-Learning for Natural Language Understanding Tasks in Finance

arXiv: https://arxiv.org/abs/2303.02841 · HTML: https://mlsanity.com/paper/2303.02841/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 6 Mar 2023.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1901.11504/) (2019)
- [Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts](https://mlsanity.com/paper/1307.5336/) (2013)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [Meta-Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1808.08437/) (2018)
- [Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Prediction](https://mlsanity.com/paper/1712.02136/) (2017)
- [Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks](https://mlsanity.com/paper/1908.10423/) (2019)
- [Bayesian multi-domain learning for cancer subtype discovery from next-generation sequencing count data](https://mlsanity.com/paper/1810.09433/) (2018)

## Built on by (most cited first, 2 of 2)

- [On-ramp and Off-ramp Traffic Flows Estimation Based on A Data-driven Transfer Learning Framework](https://mlsanity.com/paper/2308.03538/) (2023)
- [A Transfer Learning Framework for Proactive Ramp Metering Performance Assessment](https://mlsanity.com/paper/2308.03542/) (2023)

## Abstract (arXiv)

Natural language understanding(NLU) is challenging for finance due to the lack of annotated data and the specialized language in that domain. As a result, researchers have proposed to use pre-trained language model and multi-task learning to learn robust representations. However, aggressive fine-tuning often causes over-fitting and multi-task learning may favor tasks with significantly larger amounts data, etc. To address these problems, in this paper, we investigate model-agnostic meta-learning algorithm(MAML) in low-resource financial NLU tasks. Our contribution includes: 1. we explore the performance of MAML method with multiple types of tasks: GLUE datasets, SNLI, Sci-Tail and Financial PhraseBank; 2. we study the performance of MAML method with multiple single-type tasks: a real scenario stock price prediction problem with twitter text data. Our models achieve the state-of-the-art performance according to the experimental results, which demonstrate that our method can adapt fast and well to low-resource situations.