mlsanityData to 14 Oct 2026

Recognized but Not Produced: A Generation Benchmark for Culturally Specific Kinship Terms

arXiv 2609.26942 ↗First posted 22 Sep 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Current literature evaluates large language models (LLMs) on multilingual kinship understanding using multiple choice benchmarks, treating it as a recognition problem. We instead prompt five open weight LLMs to generate kinship terms in three non Western languages (Hindi, Tamil, and Korean) across two communicative tasks and pair this with a matched option-supported selection baseline. On identical relation language cells, GPT OSS120B selects the correct term in 90.67% of 75 valid cells but produces an accepted term in 36.00% of the corresponding attempts; Llama 3.370B shows the same pattern (77.92% versus 24.24%). Since the four-option condition displays the candidate terms and does not require script production, the difference is interpreted as an evaluation format gap rather than direct proof that lexical knowledge is intact. On explicitly specified L3 prompts, accuracy varies sharply, from GLM-5.1 at 72.29% to Llama-3.370B at 24.24%. The paternal-lineage advantage is language specific; it is large in Hindi but weak or reversed in Korean, while Tamil shared-term pairs provide a control for measurement variation. These results show that culturally specific kinship generation remains difficult even when the relationship is explicitly stated and motivate generation-based evaluation alongside multiple-choice testing.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 11 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

7 of 11 references.

References · 7

  1. 2023Empowering Cross-lingual Abilities of Instruction-tuned Large Language Models by Translation-following demonstrationscited by 35
  2. 2024RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanizationcited by 29
  3. 2024Towards Cross-Cultural Machine Translation with Retrieval-Augmented Generation from Multilingual Knowledge Graphscited by 22
  4. 2022Using Linguistic Typology to Enrich Multilingual Lexicons: the Case of Lexical Gaps in Kinshipcited by 9
  5. 2024Low-resource neural machine translation with morphological modelingcited by 7
  6. 2026Culture-Aware Machine Translation in Large Language Models: Benchmarking and Investigationcited by 4
  7. 2026The Future of Facts: Tracing the Factual Generation-Verification Gapcited by 1

Built on by

Not cited yet.