mlsanityData to 7 Oct 2026

KemenkeuGPT: Leveraging a Large Language Model on Indonesia's Government Financial Data and Regulations to Enhance Decision Making

arXiv 2407.21459 ↗First posted 31 Jul 2024cs.AIcited by 0

On the map: Retrieval-augmented generation → Legal and financial RAG

Abstract (arXiv)

Data is crucial for evidence-based policymaking and enhancing public services, including those at the Ministry of Finance of the Republic of Indonesia. However, the complexity and dynamic nature of governmental financial data and regulations can hinder decision-making. This study investigates the potential of Large Language Models (LLMs) to address these challenges, focusing on Indonesia's financial data and regulations. While LLMs are effective in the financial sector, their use in the public sector in Indonesia is unexplored. This study undertakes an iterative process to develop KemenkeuGPT using the LangChain with Retrieval-Augmented Generation (RAG), prompt engineering and fine-tuning. The dataset from 2003 to 2023 was collected from the Ministry of Finance, Statistics Indonesia and the International Monetary Fund (IMF). Surveys and interviews with Ministry officials informed, enhanced and fine-tuned the model. We evaluated the model using human feedback, LLM-based evaluation and benchmarking. The model's accuracy improved from 35% to 61%, with correctness increasing from 48% to 64%. The Retrieval-Augmented Generation Assessment (RAGAS) framework showed that KemenkeuGPT achieved 44% correctness with 73% faithfulness, 40% precision and 60% recall, outperforming several other base models. An interview with an expert from the Ministry of Finance indicated that KemenkeuGPT has the potential to become an essential tool for decision-making. These results are expected to improve with continuous human feedback.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Legal and financial RAG, one of 15 directions in Retrieval-augmented generation. Builds on 12 of its 28 references that are in the map. First posted on arXiv 31 Jul 2024.

Builds on

12 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models6 Oct

Other references · 11

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2023A Survey of Large Language Modelscited by 2,537
  6. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  7. 2023BloombergGPT: A Large Language Model for Financecited by 923
  8. 2022Galactica: A Large Language Model for Sciencecited by 736
  9. 2023FinGPT: Democratizing Internet-scale Data for Financial Large Language Modelscited by 65
  10. 2020How fine can fine-tuning be? Learning efficient language modelscited by 41
  11. 2019IMHO Fine-Tuning Improves Claim Detectioncited by 27

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024SusGen-GPT: A Data-Centric LLM for Financial NLP and Sustainability Report Generation
  2. 2024OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain
  3. 2025FinS-Pilot: A Benchmark for Online Financial RAG System
  4. 2024Multi-Reranker: Maximizing performance of retrieval-augmented generation in the FinanceRAG challenge
  5. 2026IndiaFinBench: An Evaluation Benchmark for Large Language Model Performance on Indian Financial Regulatory Text
  6. 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU
  7. 2025A Greek Government Decisions Dataset for Public-Sector Analysis and Insight
  8. 2025Finance Language Model Evaluation (FLaME)