Pancasila-Dilemmas: Evaluating Large Language Models on Indonesian Human Value Dilemmas Grounded in Pancasila
On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment
Abstract (arXiv)
The value alignment of large language models (LLMs) is crucial for ensuring responses align with human intention and value preferences. However, most evaluations of value alignment focus on Western or universal values, while assessments grounded in the value systems of specific countries remain scarce. In this paper, we introduce Pancasila-Dilemmas, an evaluation dataset of 1,834 questions derived from Indonesian news, classified by 5 values of Pancasila: Religion, Humanity, Unity, Democracy, and Social Justice. This dataset reflects daily life in Indonesia, making it suitable for measuring the value alignment of LLMs deployed for Indonesia. To ensure a more rigorous evaluation, we choose scenarios containing dilemmas. The dataset is proofread by native speakers and answered by 5 diverse Indonesian citizens. We evaluate 50 closed- and open-source LLMs on our dataset. Results reveal that all evaluated LLMs achieves less than 0.5 Probability Match Score (PMS) and 0.72 Max-Vote Agreement Score (MVAS). Compared by each values, LLMs mostly struggle in Religion and Unity dilemma cases. This highlights a significant gap in capturing Indonesian values. The dataset is publicly available at https://github.com/tjunlp-lab/Pancasila-Dilemmas.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 8 of its 37 references that are in the map. First posted on arXiv 20 Jul 2026.
Builds on
8 of 37 references.
Closest ancestors · 1
Same direction, newest first
- 2025C-VARC: A Large-Scale Chinese Value Rule Corpus for Value Alignment of Large Language Models2 Jun
Other references · 7
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 123
- 2025SEA-LION: Southeast Asian Languages in One Networkcited by 22
- 2026Tiny Aya: Bridging Scale and Multilingual Depthcited by 15
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Benchmarking Multi-National Value Alignment for Large Language Models
- 2026LKValues: Aligning Large Language Models with Sri Lankan Societal Values
- 2026PLURAL: A Global Dataset for Value Alignment
- 2025CCD-Bench: Probing Cultural Conflict in Large Language Model Decision-Making
- 2026Mind the Gap: Pitfalls of LLM Alignment with Asian Public Opinion
- 2025On the Alignment of Large Language Models with Global Human Opinion
- 2023Heterogeneous Value Alignment Evaluation for Large Language Models
- 2026People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe