mlsanityData to 7 Oct 2026

Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training

arXiv 2502.17800 ↗First posted 25 Feb 2025cs.CLcited by 3

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated strong reasoning capabilities across various tasks. However, even minor variations in query phrasing, despite preserving the underlying semantic meaning, can significantly affect their performance. To address this, we focus on enhancing LLMs' awareness of symmetry in query variations and propose syMmetry-ENhanceD (MEND) Data Augmentation, a data-centric approach that improves the model's ability to extract useful information from context. Unlike existing methods that emphasize reasoning chain augmentation, our approach improves model robustness at the knowledge extraction stage through query augmentations, enabling more data-efficient training and stronger generalization to Out-of-Distribution (OOD) settings. Extensive experiments on both logical and arithmetic reasoning tasks show that MEND enhances reasoning performance across diverse query variations, providing new insight into improving LLM robustness through structured dataset curation.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 32 of its 48 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

32 of 48 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models23 Oct
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct

Other references · 30

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Let's Verify Step by Stepcited by 3,598
  5. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  6. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  7. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  8. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  9. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  10. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 276
  11. 2023Planning with Large Language Models for Code Generationcited by 179
  12. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 162
18 more
  1. 2024Rho-1: Not All Tokens Are What You Needcited by 132
  2. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 110
  3. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 108
  4. 2024Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Processcited by 107
  5. 2022SO(2)-Equivariant Reinforcement Learningcited by 79
  6. 2024EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learningcited by 72
  7. 2024Premise Order Matters in Reasoning with Large Language Modelscited by 71
  8. 2024Augmenting Math Word Problems via Iterative Question Composingcited by 58
  9. 2024Dynamic Evaluation of Large Language Models by Meta Probing Agentscited by 54
  10. 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Searchcited by 53
  11. 2024Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problemscited by 49
  12. 2024Reverse Training to Nurse the Reversal Cursecited by 48
  13. 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Searchcited by 38
  14. 2024Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agentscited by 35
  15. 2024Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanismcited by 15
  16. 2024Paraphrase and Solve: Exploring and Exploiting the Impact of Surface Form on Mathematical Reasoning in Large Language Modelscited by 14
  17. 2024Exploring the Mystery of Influential Data for Mathematical Reasoningcited by 13
  18. 2024Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokenscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Behavior Injection: Preparing Language Models for Reinforcement Learningcited by 8
  2. 2026CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoningcited by 0
  3. 2026Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Computecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training
  2. 2025Multidimensional Consistency Improves Reasoning in Language Models
  3. 2024Exploring Group and Symmetry Principles in Large Language Models
  4. 2025Leveraging Parameter Space Symmetries for Reasoning Skill Transfer in LLMs
  5. 2024Calibrating Reasoning in Language Models with Internal Consistency
  6. 2025Out-of-Context Reasoning in Large Language Models
  7. 2026EvalQReason: A Framework for Step-Level Reasoning Evaluation in Large Language Models
  8. 2023MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning