mlsanityData to 7 Oct 2026

Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering

arXiv 2210.01613 ↗First posted 4 Oct 2022cs.CLcited by 71

Not on the map. Why?

Abstract (arXiv)

We introduce Mintaka, a complex, natural, and multilingual dataset designed for experimenting with end-to-end question-answering models. Mintaka is composed of 20,000 question-answer pairs collected in English, annotated with Wikidata entities, and translated into Arabic, French, German, Hindi, Italian, Japanese, Portuguese, and Spanish for a total of 180,000 samples. Mintaka includes 8 types of complex questions, including superlative, intersection, and multi-hop questions, which were naturally elicited from crowd workers. We run baselines over Mintaka, the best of which achieves 38% hits@1 in English and 31% hits@1 multilingually, showing that existing models have room for improvement. We release Mintaka at https://github.com/amazon-research/mintaka.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 23 of its 39 references that are in the map. First posted on arXiv 4 Oct 2022.

Builds on

23 of 39 references.

References · 23

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  8. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  9. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  10. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  11. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  12. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
11 more
  1. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  2. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  3. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  4. 2016Key-Value Memory Networks for Directly Reading Documentscited by 223
  5. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  6. 2020Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Basescited by 169
  7. 2021A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutionscited by 77
  8. 2021TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graphcited by 60
  9. 2020Scalable Neural Methods for Reasoning With a Symbolic Knowledge Basecited by 32
  10. 2021Expanding End-to-End Question Answering on Differentiable Knowledge Graphs with Intersectioncited by 12
  11. 2021End-to-End Entity Resolution and Question Answering Using Differentiable Knowledge Graphscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 23; 2025: 28; 2026: 9.

71 papers.

Sort:

By area:

  1. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  2. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  3. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  4. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 180
  5. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  6. 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
  7. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  8. 2023Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofittingcited by 70
  9. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  10. 2023Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answeringcited by 64
  11. 2025Enhancing Multilingual LLM Pretraining with Model-Based Data Selectioncited by 33
  12. 2025MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languagescited by 22
38 more
  1. 2023Knowledge-Augmented Language Model Verificationcited by 21
  2. 2024Towards Cross-Cultural Machine Translation with Retrieval-Augmented Generation from Multilingual Knowledge Graphscited by 21
  3. 2024Atomic Fact Decomposition Helps Attributed Question Answeringcited by 21
  4. 2024Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddingscited by 20
  5. 2024MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledgecited by 20
  6. 2023DBLP-QuAD: A Question Answering Dataset over the DBLP Scholarly Knowledge Graphcited by 18
  7. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Surveycited by 18
  8. 2024An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaborationcited by 16
  9. 2025Knowledge Graph-extended Retrieval Augmented Generation for Question Answeringcited by 14
  10. 2024SMARTCAL: An Approach to Self-Aware Tool-Use Evaluation and Calibrationcited by 13
  11. 2024Retrieving, Rethinking and Revising: The Chain-of-Verification Can Improve Retrieval Augmented Generationcited by 12
  12. 2023Large Language Models Meet Knowledge Graphs to Answer Factoid Questionscited by 11
  13. 2024Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answeringcited by 9
  14. 2025Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translationcited by 8
  15. 2025Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuningcited by 8
  16. 2025Granite Embedding Modelscited by 7
  17. 2025LLM-Independent Adaptive RAG: Let the Question Speak for Itselfcited by 7
  18. 2026SPADER: Step-wise Peer Advantage with Diversity-Aware Exploration Rewards for Multi-Answer Question Answeringcited by 6
  19. 2024PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguesecited by 5
  20. 2024DEXTER: A Benchmark for open-domain Complex Question Answering using LLMscited by 4
  21. 2024GRAF: Graph Retrieval Augmented by Facts for Romanian Legal Multi-Choice Question Answeringcited by 4
  22. 2025CoCo-CoLa: Evaluating and Improving Language Adherence in Multilingual LLMscited by 4
  23. 2025MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMscited by 4
  24. 2025Aligning Knowledge Graphs and Language Models for Factual Accuracycited by 4
  25. 2025Prompting Large Language Models with Partial Knowledge for Answering Questions with Unseen Entitiescited by 4
  26. 2024Konstruktor: A Strong Baseline for Simple Knowledge Graph Question Answeringcited by 3
  27. 2023Answer Candidate Type Selection: Text-to-Text Language Model for Closed Book Question Answering Meets Knowledge Graphscited by 2
  28. 2023GraphextQA: A Benchmark for Evaluating Graph-Enhanced Large Language Modelscited by 2
  29. 2024Pyramid-Driven Alignment: Pyramid Principle Guided Integration of Large Language Models and Knowledge Graphscited by 2
  30. 2025RoToR: Towards More Reliable Responses for Order-Invariant Inputscited by 2
  31. 2025MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM Hallucinationscited by 2
  32. 2025Walk&Retrieve: Simple Yet Effective Zero-shot Retrieval-Augmented Generation via Knowledge Graph Walkscited by 2
  33. 2025Decomposing and Revising What Language Models Generatecited by 2
  34. 2025Quantifying Genuine Awareness in Hallucination Prediction Beyond Question-Side Shortcutscited by 2
  35. 2025Ground-Truth Subgraphs for Better Training and Evaluation of Knowledge Graph Augmented LLMscited by 2
  36. 2026JobResQA: A Benchmark for LLM Machine Reading Comprehension on Multilingual R'esum'es and JDscited by 2
  37. 2026Masked by Consensus: Disentangling Privileged Knowledge in LLM Correctnesscited by 2
  38. 2024Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuningcited by 1

50 most cited of 71 All 71 (JSON)