mlsanityData to 7 Oct 2026

Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning

arXiv 2209.14610 ↗First posted 29 Sep 2022cs.LG cs.AI cs.CLcited by 347

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Mathematical reasoning, a core ability of human intelligence, presents unique challenges for machines in abstract thinking and logical reasoning. Recent large pre-trained language models such as GPT-3 have achieved remarkable progress on mathematical reasoning tasks written in text form, such as math word problems (MWP). However, it is unknown if the models can handle more complex problems that involve math reasoning over heterogeneous information, such as tabular data. To fill the gap, we present Tabular Math Word Problems (TabMWP), a new dataset containing 38,431 open-domain grade-level problems that require mathematical reasoning on both textual and tabular data. Each question in TabMWP is aligned with a tabular context, which is presented as an image, semi-structured text, and a structured table. There are two types of questions: free-text and multi-choice, and each problem is annotated with gold solutions to reveal the multi-step reasoning process. We evaluate different pre-trained models on TabMWP, including the GPT-3 model in a few-shot setting. As earlier studies suggest, since few-shot GPT-3 relies on the selection of in-context examples, its performance is unstable and can degrade to near chance. The unstable issue is more severe when handling complex problems like TabMWP. To mitigate this, we further propose a novel approach, PromptPG, which utilizes policy gradient to learn to select in-context examples from a small amount of training data and then constructs the corresponding prompt for the test example. Experimental results show that our method outperforms the best baseline by 5.31% on the accuracy metric and reduces the prediction variance significantly compared to random selection, which verifies its effectiveness in selecting in-context examples.

Key facts. Cited by 347 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 45 of its 58 references that are in the map. First posted on arXiv 29 Sep 2022.

Builds on

45 of 58 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  9. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  11. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  12. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
33 more
  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  4. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  5. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  6. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  7. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  8. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  9. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  10. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  11. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  12. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  13. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  14. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  15. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  16. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  17. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 376
  18. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  19. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  20. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  21. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 262
  22. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  23. 2018Distributed Distributional Deterministic Policy Gradientscited by 195
  24. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 177
  25. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  26. 2020Open Question Answering over Tables and Textcited by 163
  27. 2022MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Datacited by 120
  28. 2021AIT-QA: Question Answering Dataset over Complex Tables in the Airline Industrycited by 44
  29. 2021ValueNet: A New Dataset for Human Value Driven Dialogue Systemcited by 34
  30. 2016Annotating Derivations: A New Evaluation Strategy and Dataset for Algebra Word Problemscited by 28
  31. 2016Unit Dependency Graph and its Application to Arithmetic Word Problem Solvingcited by 25
  32. 2016TabMCQ: A Dataset of General Knowledge Tables and Multiple-choice Questionscited by 18
  33. 2017Batch Policy Gradient Methods for Improving Neural Conversation Modelscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 58; 2024: 118; 2025: 113; 2026: 52.

347 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  4. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  5. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  6. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  7. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  8. 2023A Survey on Multimodal Large Language Modelscited by 898
  9. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  10. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  11. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  12. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
38 more
  1. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  2. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  3. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  4. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  5. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  6. 2024What matters when building vision-language models?cited by 321
  7. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  8. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  9. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  10. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
  11. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 276
  12. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273
  13. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  14. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
  15. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 244
  16. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 214
  17. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  18. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 195
  19. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  20. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
  21. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 189
  22. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
  23. 2024Large Language Models for Education: A Survey and Outlookcited by 185
  24. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  25. 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 174
  26. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 171
  27. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  28. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  29. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 156
  30. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 144
  31. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Datasetcited by 143
  32. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 140
  33. 2022UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expressioncited by 137
  34. 2024A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalistcited by 137
  35. 2024InfiAgent-DABench: Evaluating Agents on Data Analysis Taskscited by 134
  36. 2024Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Modelscited by 133
  37. 2024Rho-1: Not All Tokens Are What You Needcited by 132
  38. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129

50 most cited of 347 All 347 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025TabularMath: Understanding Math Reasoning over Tables with Large Language Models
  2. 2025A Survey on Large Language Models for Mathematical Reasoning
  3. 2026MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval
  4. 2024Benchmarking Large Language Models for Math Reasoning Tasks
  5. 2022Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions
  6. 2023Reasoning in Large Language Models Through Symbolic Math Word Problems
  7. 2025JT-Math: A Multi-Stage Framework for Advanced Mathematical Reasoning in Large Language Models
  8. 2024Expanding Search Space with Diverse Prompting Agents: An Efficient Sampling Approach for LLM Mathematical Reasoning