mlsanityData to 14 Oct 2026

OMoS-QA: A Dataset for Cross-Lingual Extractive Question Answering in a German Migration Context

arXiv 2407.15736 ↗First posted 22 Jul 2024cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

When immigrating to a new country, it is easy to feel overwhelmed by the need to obtain information on financial support, housing, schooling, language courses, and other issues. If relocation is rushed or even forced, the necessity for high-quality answers to such questions is all the more urgent. Official immigration counselors are usually overbooked, and online systems could guide newcomers to the requested information or a suitable counseling service. To this end, we present OMoS-QA, a dataset of German and English questions paired with relevant trustworthy documents and manually annotated answers, specifically tailored to this scenario. Questions are automatically generated with an open-source large language model (LLM) and answer sentences are selected by crowd workers with high agreement. With our data, we conduct a comparison of 5 pretrained LLMs on the task of extractive question answering (QA) in German and English. Across all models and both languages, we find high precision and low-to-mid recall in selecting answer sentences, which is a favorable trade-off to avoid misleading users. This performance even holds up when the question language does not match the document language. When it comes to identifying unanswerable questions given a context, there are larger differences between the two languages.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 19 of its 52 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

19 of 52 references.

References · 19

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  4. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  6. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  7. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  8. 2021Generating Datasets with Pretrained Language Modelscited by 193
  9. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 169
  10. 2017Simple and Effective Multi-Paragraph Reading Comprehensioncited by 120
  11. 2017Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisiscited by 96
  12. 2021GermanQuAD and GermanDPR: Improving Non-English Question Answering and Passage Retrievalcited by 39
7 more
  1. 2019Controlled Crowdsourcing for High-Quality QA-SRL Annotationcited by 34
  2. 2022Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generationcited by 17
  3. 2021QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositionscited by 16
  4. 2019Evidence Sentence Extraction for Machine Reading Comprehensioncited by 14
  5. 2022Choose Your QA Model Wisely: A Systematic Study of Generative and Extractive Readers for Question Answeringcited by 11
  6. 2022Knowledge Graph - Deep Learning: A Case Study in Question Answering in Aviation Safety Domaincited by 5
  7. 2019Propagate-Selector: Detecting Supporting Sentences for Question Answering via Graph Neural Networkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Towards Empowering Consumers through Sentence-level Readability Scoring in German ESG Reportscited by 0