Benchmarking Source-Sensitive Reasoning in Turkish: Humans and LLMs under Evidential Trust Manipulation
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
This paper investigates whether source trustworthiness shapes Turkish evidential morphology and whether large language models (LLMs) track this sensitivity. We study the past-domain contrast between -DI and -mIs in controlled cloze contexts where the information source is overtly external, while only its perceived reliability is manipulated (High-Trust vs. Low-Trust). In a human production experiment, native speakers of Turkish show a robust trust effect: High-Trust contexts yield relatively more -DI, whereas Low-Trust contexts yield relatively more -mIs, with the pattern remaining stable across sensitivity analyses. We then evaluate 10 LLMs in three prompting paradigms (open gap-fill, explicit past-tense gap-fill, and forced-choice A/B selection). LLM behavior is highly model- and prompt-dependent: some models show weak or local trust-consistent shifts, but effects are generally unstable, often reversed, and frequently overshadowed by output-compliance problems and strong base-rate suffix preferences. The results provide new evidence for a trust-/commitment-based account of Turkish evidentiality and reveal a clear human-LLM gap in source-sensitive evidential reasoning.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 16 of its 64 references that are in the map. First posted on arXiv 27 Apr 2026.
Builds on
16 of 64 references.
Closest ancestors · 3
Same direction, newest first
- 2026Plausibility as Commonsense Reasoning: Humans Succeed, Large Language Models Do not6 Apr
- 2026Clause-Internal or Clause-External? Testing Turkish Reflexive Binding in Adapted versus Chain of Thought Large Language Models30 Jan
- 2025Qwen3 Technical Report14 May
Other references · 13
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
- 2025Gemma 3 Technical Reportcited by 1,686
- 2025OpenAI GPT-5 System Cardcited by 910
- 2025HalluLens: LLM Hallucination Benchmarkcited by 88
- 2026AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agentscited by 15
- 2025Tokenization Standards for Linguistic Integrity: Turkish as a Benchmarkcited by 8
- 2025Tokens with Meaning: A Hybrid Tokenization Approach for Turkishcited by 7
- 2025Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkishcited by 7
- 2026Revisiting Prompt Sensitivity in Large Language Models for Text Classification: The Role of Prompt Underspecificationcited by 3
- 2026Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domaincited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts
- 2026Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs
- 2026Rethinking Uncertainty Evaluation in Large Language Models
- 2026An Analysis of Training-Free Self-Reported Confidence in Language Models
- 2023Establishing Trustworthiness: Rethinking Tasks and Model Evaluation
- 2026Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models
- 2026Trust, but Don't Verify: Epistemic Blind Spots in LLM Source Evaluation
- 2026Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models