mlsanityData to 7 Oct 2026

MULTITAT: Benchmarking Multilingual Table-and-Text Question Answering

arXiv 2502.17253 ↗First posted 24 Feb 2025cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Question answering on the hybrid context of tables and text (TATQA) is a critical task, with broad applications in data-intensive domains. However, existing TATQA datasets are limited to English, leading to several drawbacks: (i) They overlook the challenges of multilingual TAT-QA and cannot assess model performance in the multilingual setting. (ii) They do not reflect real-world scenarios where tables and texts frequently appear in non-English languages. To address the limitations, we propose the first multilingual TATQA dataset (MULTITAT). Specifically, we sample data from 3 mainstream TATQA datasets and translate it into 10 diverse languages. To align the model TATQA capabilities in English with other languages, we develop a baseline, Ours. Experimental results reveal that the performance on non-English data in MULTITAT drops by an average of 19.4% compared to English, proving the necessity of MULTITAT. We further analyze the reasons for this performance gap. Furthermore, Ours outperforms other baselines by an average of 3.3, demonstrating its effectiveness.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 39 of its 58 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

39 of 58 references.

References · 39

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  4. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  5. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  6. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  7. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  8. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  9. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  10. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  11. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  12. 2020WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarizationcited by 154
27 more
  1. 2020MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answeringcited by 151
  2. 2020MLSUM: The Multilingual Summarization Corpuscited by 121
  3. 2024HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalizationcited by 84
  4. 2024Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Datacited by 61
  5. 2024IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languagescited by 55
  6. 2023DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documentscited by 45
  7. 2023FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domainscited by 39
  8. 2020A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamesecited by 36
  9. 2019A Pilot Study for Chinese SQL Semantic Parsingcited by 33
  10. 2024P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMscited by 29
  11. 2024BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebracited by 23
  12. 2022MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsingcited by 22
  13. 2024GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translatorscited by 22
  14. 2021TSQA: Tabular Scenario Based Question Answeringcited by 21
  15. 2024GPT-4 vs. Human Translators: A Comprehensive Evaluation of Translation Quality Across Languages, Domains, and Expertise Levelscited by 20
  16. 2023ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessmentcited by 19
  17. 2022Answering Numerical Reasoning Questions in Table-Text Hybrid Contents with Graph-based Encoder and Tree-based Decodercited by 17
  18. 2023S3HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answeringcited by 17
  19. 2024Enhancing Financial Question Answering with a Multi-Agent Reflection Frameworkcited by 12
  20. 2024SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Typescited by 10
  21. 2024Exploring Hybrid Question Answering via Program-based Promptingcited by 8
  22. 2024Eliciting Better Multilingual Structured Reasoning from LLMs through Codecited by 8
  23. 2024clembench-2024: A Challenging, Dynamic, Complementary, Multilingual Benchmark and Underlying Flexible Framework for LLMs as Multi-Action Agentscited by 8
  24. 2024An Open Multilingual System for Scoring Readability of Wikipediacited by 7
  25. 2024TTQA-RS- A break-down prompting approach for Multi-hop Table-Text Question Answering with Reasoning and Summarizationcited by 6
  26. 2023XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representationscited by 5
  27. 2023HRoT: Hybrid prompt strategy and Retrieval of Thought for Table-Text Hybrid Question Answeringcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025M3TQA: Massively Multilingual Multitask Table Question Answeringcited by 3
  2. 2026An Actionable Diagnosis of Multilingual, Multi-Agent Planning Failurescited by 0