# Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance

arXiv: https://arxiv.org/abs/2502.18772 · HTML: https://mlsanity.com/paper/2502.18772/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 47 references that are in the map.
- First posted on arXiv 26 Feb 2025.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants](https://mlsanity.com/paper/2308.16884/) (2023)
- [FinanceBench: A New Benchmark for Financial Question Answering](https://mlsanity.com/paper/2311.11944/) (2023)
- [PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance](https://mlsanity.com/paper/2306.05443/) (2023)
- [InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuning](https://mlsanity.com/paper/2309.13064/) (2023)
- [FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets](https://mlsanity.com/paper/2310.04793/) (2023)
- [DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuning](https://mlsanity.com/paper/2310.15205/) (2023)
- [FiNER-ORD: Financial Named Entity Recognition Open Research Dataset](https://mlsanity.com/paper/2302.11157/) (2023)
- [Meltemi: The first open Large Language Model for Greek](https://mlsanity.com/paper/2407.20743/) (2024)
- [NumHTML: Numeric-Oriented Hierarchical Transformer Model for Multi-task Financial Forecasting](https://mlsanity.com/paper/2201.01770/) (2022)
- [Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Tasks](https://mlsanity.com/paper/2305.16633/) (2023)
- [NLP for The Greek Language: A Longer Survey](https://mlsanity.com/paper/2408.10962/) (2024)
- [Numeral Understanding in Financial Tweets for Fine-grained Crowd-based Forecasting](https://mlsanity.com/paper/1809.05356/) (2018)

## Built on by (most cited first, 18 of 18)

- [When Agents Trade: Live Multi-Market Trading Benchmark for LLM Agents](https://mlsanity.com/paper/2510.11695/) (2025)
- [FinCARDS: Card-Based Analyst Reranking for Financial Document Question Answering](https://mlsanity.com/paper/2601.06992/) (2026)
- [GreekBarBench: A Challenging Benchmark for Free-Text Legal Reasoning and Citations](https://mlsanity.com/paper/2505.17267/) (2025)
- [MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application](https://mlsanity.com/paper/2506.14028/) (2025)
- [GreekMMLU: A Native-Sourced Multitask Benchmark for Evaluating Language Models in Greek](https://mlsanity.com/paper/2602.05150/) (2026)
- [Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training](https://mlsanity.com/paper/2603.07223/) (2026)
- [FinTagging: Benchmarking LLMs for Extracting and Structuring Financial Information](https://mlsanity.com/paper/2505.20650/) (2025)
- [INSEva: A Comprehensive Chinese Benchmark for Large Language Models in Insurance](https://mlsanity.com/paper/2509.04455/) (2025)
- [The CLEF-2026 FinMMEval Lab: Multilingual and Multimodal Evaluation of Financial AI Systems](https://mlsanity.com/paper/2602.10886/) (2026)
- [Evaluating Monolingual and Multilingual Large Language Models for Greek Question Answering: The DemosQA Benchmark](https://mlsanity.com/paper/2602.16811/) (2026)
- [SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoning](https://mlsanity.com/paper/2604.19098/) (2026)
- [Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documents](https://mlsanity.com/paper/2608.12342/) (2026)
- [RKEFino1: A Regulation Knowledge-Enhanced Large Language Model](https://mlsanity.com/paper/2506.05700/) (2025)
- [Diagnosing Translated Benchmarks: An Automated Quality Assurance Study of the EU20 Benchmark Suite](https://mlsanity.com/paper/2604.01957/) (2026)
- [TaxPraBen: A Scalable Benchmark for Structured Evaluation of LLMs in Chinese Real-World Tax Practice](https://mlsanity.com/paper/2604.08948/) (2026)
- [Pass or Fail? Evaluating LLMs on Two Greek Examination Benchmarks](https://mlsanity.com/paper/2609.34800/) (2026)
- [Fin_Agent at FinEval 2026: Task-Aware Methods for Multilingual Financial NLP](https://mlsanity.com/paper/2610.14101/) (2026)
- [Recognition Is Not Grounding: Benchmarking Multilingual Financial Evidence Extraction](https://mlsanity.com/paper/2610.19432/) (2026)

## Abstract (arXiv)

Despite Greece's pivotal role in the global economy, large language models (LLMs) remain underexplored for Greek financial context due to the linguistic complexity of Greek and the scarcity of domain-specific datasets. Previous efforts in multilingual financial natural language processing (NLP) have exposed considerable performance disparities, yet no dedicated Greek financial benchmarks or Greek-specific financial LLMs have been developed until now. To bridge this gap, we introduce Plutus-ben, the first Greek Financial Evaluation Benchmark, and Plutus-8B, the pioneering Greek Financial LLM, fine-tuned with Greek domain-specific data. Plutus-ben addresses five core financial NLP tasks in Greek: numeric and textual named entity recognition, question answering, abstractive summarization, and topic classification, thereby facilitating systematic and reproducible LLM assessments. To underpin these tasks, we present three novel, high-quality Greek financial datasets, thoroughly annotated by expert native Greek speakers, augmented by two existing resources. Our comprehensive evaluation of 22 LLMs on Plutus-ben reveals that Greek financial NLP remains challenging due to linguistic complexity, domain-specific terminology, and financial reasoning gaps. These findings underscore the limitations of cross-lingual transfer, the necessity for financial expertise in Greek-trained models, and the challenges of adapting financial LLMs to Greek text. We release Plutus-ben, Plutus-8B, and all associated datasets publicly to promote reproducible research and advance Greek financial NLP, fostering broader multilingual inclusivity in finance.