mlsanityData to 14 Oct 2026

MAPLE: Metadata Conditioned LLM Pretraining for Locale-Aware Question Answering

arXiv 2601.15236 ↗First posted 21 Jan 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models can memorize competing locale-specific facts yet fail to select among them when the locale changes, defaulting instead to a single globally dominant answer. We formalize this as localized knowledge disambiguation and introduce LocalNewsQA, an 18,700-item English-news benchmark that pairs the same question across two locales and scores whether a model actually switches its answer when the locale changes. We also introduce MAPLE, a controlled family of decoder-only models pretrained with document-level geographic metadata (source URL, country, and continent) already present in the training corpus, and compare it to metadata-free controls trained on identical data with the same token budget, architecture, and optimization. In controlled experiments at 1B and 3B, with inference-time metadata fixed, pretraining with metadata in MAPLE produces measurable switching and improves accuracy on questions whose correct answer depends on locale. Ablations and external-benchmark evaluations further suggest that locale-conditioned prediction benefits from geographic provenance learned during pretraining and that these benefits strengthen at larger model sizes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 21 Jan 2026.

Builds on

25 of 33 references.

References · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  4. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  5. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  6. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  7. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  8. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  9. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  10. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  11. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 398
  12. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 340
13 more
  1. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
  2. 2023Pretraining Language Models with Human Preferencescited by 247
  3. 2024Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluationcited by 191
  4. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 173
  5. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 167
  6. 2020MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answeringcited by 153
  7. 2025Specializing Large Language Models to Simulate Survey Response Distributions for Global Populationscited by 53
  8. 2024NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Modelscited by 48
  9. 2023"According to ...": Prompting Language Models Improves Quoting from Pre-Training Datacited by 42
  10. 2024Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approachcited by 26
  11. 2025Metadata Conditioning Accelerates Language Model Pre-trainingcited by 22
  12. 2021LMSOC: An Approach for Socially Sensitive Pretrainingcited by 8
  13. 2022NTULM: Enriching Social Media Text Representations with Non-Textual Unitscited by 2

Built on by

Not cited yet.