# Investigating Math Word Problems using Pretrained Multilingual Language Models

arXiv: https://arxiv.org/abs/2105.08928 · HTML: https://mlsanity.com/paper/2105.08928/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms](https://mlsanity.com/paper/1905.13319/) (2019)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [Solving General Arithmetic Word Problems](https://mlsanity.com/paper/1608.01413/) (2016)
- [MLQA: Evaluating Cross-lingual Extractive Question Answering](https://mlsanity.com/paper/1910.07475/) (2019)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://mlsanity.com/paper/1904.09077/) (2019)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System](https://mlsanity.com/paper/1802.08979/) (2018)
- [Translating a Math Word Problem to an Expression Tree](https://mlsanity.com/paper/1811.05632/) (2018)
- [Program Synthesis using Natural Language](https://mlsanity.com/paper/1509.00413/) (2015)
- [Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems](https://mlsanity.com/paper/2010.06823/) (2020)
- [Program Synthesis and Semantic Parsing with Learned Code Idioms](https://mlsanity.com/paper/1906.10816/) (2019)
- [Deep Neural Machine Translation with Linear Associative Unit](https://mlsanity.com/paper/1705.00861/) (2017)

## Built on by (most cited first, 21 of 21)

- [ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification](https://mlsanity.com/paper/2405.14256/) (2024)
- [Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction](https://mlsanity.com/paper/2203.10316/) (2022)
- [MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers](https://mlsanity.com/paper/2109.00799/) (2021)
- [MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving](https://mlsanity.com/paper/2107.13435/) (2021)
- [Seeking Patterns, Not just Memorizing Procedures: Contrastive Learning for Solving Math Word Problems](https://mlsanity.com/paper/2110.08464/) (2021)
- [Learning by Analogy: Diverse Questions Generation in Math Word Problem](https://mlsanity.com/paper/2306.09064/) (2023)
- [MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning](https://mlsanity.com/paper/2307.07951/) (2023)
- [Chaining Simultaneous Thoughts for Numerical Reasoning](https://mlsanity.com/paper/2211.16482/) (2022)
- [Multi-View Reasoning: Consistent Contrastive Learning for Math Word Problem](https://mlsanity.com/paper/2210.11694/) (2022)
- [Design of Chain-of-Thought in Math Problem Solving](https://mlsanity.com/paper/2309.11054/) (2023)
- [Numeracy from Literacy: Data Science as an Emergent Skill from Large Language Models](https://mlsanity.com/paper/2301.13382/) (2023)
- [Analogical Math Word Problems Solving with Enhanced Problem-Solution Association](https://mlsanity.com/paper/2212.00837/) (2022)
- [An Expression Tree Decoding Strategy for Mathematical Equation Generation](https://mlsanity.com/paper/2310.09619/) (2023)
- [Seeking Diverse Reasoning Logic: Controlled Equation Expression Generation for Solving Math Word Problems](https://mlsanity.com/paper/2209.10310/) (2022)
- [Solving Math Word Problems with Reexamination](https://mlsanity.com/paper/2310.09590/) (2023)
- [From Large to Tiny: Distilling and Refining Mathematical Expertise for Math Word Problems with Weakly Supervision](https://mlsanity.com/paper/2403.14390/) (2024)
- [ComSearch: Equation Searching with Combinatorial Strategy for Solving Math Word Problems with Weak Supervision](https://mlsanity.com/paper/2210.07017/) (2022)
- [Textual Enhanced Contrastive Learning for Solving Math Word Problems](https://mlsanity.com/paper/2211.16022/) (2022)
- [Techniques to Improve Neural Math Word Problem Solvers](https://mlsanity.com/paper/2302.03145/) (2023)
- [NUMCoT: Numerals and Units of Measurement in Chain-of-Thought Reasoning using Large Language Models](https://mlsanity.com/paper/2406.02864/) (2024)
- [A Diversity-Enhanced Knowledge Distillation Model for Practical Math Word Problem Solving](https://mlsanity.com/paper/2501.03670/) (2025)

## Abstract (arXiv)

In this paper, we revisit math word problems~(MWPs) from the cross-lingual and multilingual perspective. We construct our MWP solvers over pretrained multilingual language models using sequence-to-sequence model with copy mechanism. We compare how the MWP solvers perform in cross-lingual and multilingual scenarios. To facilitate the comparison of cross-lingual performance, we first adapt the large-scale English dataset MathQA as a counterpart of the Chinese dataset Math23K. Then we extend several English datasets to bilingual datasets through machine translation plus human annotation. Our experiments show that the MWP solvers may not be transferred to a different language even if the target expressions have the same operator set and constants. But for both cross-lingual and multilingual cases, it can be better generalized if problem types exist on both source language and target language.