JaFIn: Japanese Financial Instruction Dataset
Not on the map. Why?
Abstract (arXiv)
We construct an instruction dataset for the large language model (LLM) in the Japanese finance domain. Domain adaptation of language models, including LLMs, is receiving more attention as language models become more popular. This study demonstrates the effectiveness of domain adaptation through instruction tuning. To achieve this, we propose an instruction tuning data in Japanese called JaFIn, the Japanese Financial Instruction Dataset. JaFIn is manually constructed based on multiple data sources, including Japanese government websites, which provide extensive financial knowledge. We then utilize JaFIn to apply instruction tuning for several LLMs, demonstrating that our models specialized in finance have better domain adaptability than the original models. The financial-specialized LLMs created were evaluated using a quantitative Japanese financial benchmark and qualitative response comparisons, showing improved performance over the originals.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 14 Apr 2024.
Builds on
30 of 39 references.
References · 30
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
18 more
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
- 2023BloombergGPT: A Large Language Model for Financecited by 939
- 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
- 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
- 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 385
- 2022The Stack: 3 TB of permissively licensed source codecited by 359
- 2023FinGPT: Open-Source Financial Large Language Modelscited by 253
- 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 222
- 2022Towards a Cleaner Document-Oriented Multilingual Crawled Corpuscited by 143
- 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 141
- 2022WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domaincited by 117
- 2024Construction of a Japanese Financial Benchmark for Large Language Modelscited by 13
- 2023llm-japanese-dataset v0: Construction of Japanese Chat Dataset for Large Language Models and its Methodologycited by 6
- 2023From Base to Conversational: Japanese Instruction Dataset and Tuning Large Language Modelscited by 5
Built on by
Citing papers per year: 2024: 2.
2 papers.