mlsanityData to 14 Oct 2026

Thai Financial Domain Adaptation of THaLLE -- Technical Report

arXiv 2411.18242 ↗First posted 27 Nov 2024cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) excel in general tasks but struggle with domain-specific challenges, such as specialized terminology and localized regulations. Existing financial LLMs, like FinGPT and BloombergGPT, lack support for the Thai financial domain. We developed a Thai Financial LLM using the Investment Consultant (IC) exam dataset from the Stock Exchange of Thailand. To address dataset limitations, we applied data augmentation, ReLoRA for efficient training, Continued Pretraining (CPT) for domain knowledge, and Rank-Stabilized LoRA (rsLoRA) for fine-tuning. Supervised Fine-Tuning (SFT) simulated exam scenarios, while Direct Preference Optimization (DPO) refined the model using feedback. The model achieved scores of 72%, 72%, and 84% on IC exam levels P1, P2, and P3, respectively, demonstrating its effectiveness in Thai financial advisory tasks and its potential for specialized applications.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 26 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

13 of 26 references.

References · 13

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2024Qwen2 Technical Reportcited by 2,120
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  6. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  7. 2023BloombergGPT: A Large Language Model for Financecited by 939
  8. 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 243
  9. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 222
  10. 2023ReLoRA: High-Rank Training Through Low-Rank Updatescited by 204
  11. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 203
  12. 2023Continual Pre-training of Language Modelscited by 198
1 more
  1. 2024SeaLLMs 3: Open Foundation and Chat Multilingual Large Language Models for Southeast Asian Languagescited by 36

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Demystifying Domain-adaptive Post-training for Financial LLMs
  2. 2024SNFinLLM: Systematic and Nuanced Financial Domain Adaptation of Chinese Large Language Models
  3. 2025LLMs Meet Finance: Fine-Tuning Foundation Models for the Open FinLLM Leaderboard
  4. 2023Beyond Classification: Financial Reasoning in State-of-the-Art Language Models
  5. 2025Large Language Models Acing Chartered Accountancy
  6. 2025QianfanHuijin Technical Report: A Novel Multi-Stage Training Paradigm for Finance Industrial LLMs
  7. 2025SPEAR-MM: Selective Parameter Evaluation and Restoration via Model Merging for Efficient Financial LLM Adaptation
  8. 2024Mixing It Up: The Cocktail Effect of Multi-Task Fine-Tuning on LLM Performance -- A Case Study in Finance