Synthetic Data Generation for Training Diversified Commonsense Reasoning Models
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Conversational agents are required to respond to their users not only with high quality (i.e. commonsense bearing) responses, but also considering multiple plausible alternative scenarios, reflecting the diversity in their responses. Despite the growing need to train diverse commonsense generators, the progress of this line of work has been significantly hindered by the lack of large-scale high-quality diverse commonsense training datasets. Due to the high annotation costs, existing Generative Commonsense Reasoning (GCR) datasets are created using a small number of human annotators, covering only a narrow set of commonsense scenarios. To address this training resource gap, we propose a two-stage method to create the first-ever synthetic dataset CommonSyn for diversified (GCR). The model fine-tuned on our synthetic data jointly increase both generation diversity and quality compared with vanilla models and the model fine-tuned on human-crafted dataset across different size Large Language Models (LLMs)
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 17 of its 59 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
17 of 59 references.
Closest ancestors · 1
Same direction, newest first
- 2025Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning26 May
Other references · 16
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
- 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
- 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
- 2019Abductive Commonsense Reasoningcited by 301
- 2024Best Practices and Lessons Learned on Synthetic Datacited by 106
- 2024Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Modelscited by 84
- 2024Comprehensive Exploration of Synthetic Data Generation: A Surveycited by 60
- 2024On the Diversity of Synthetic Data and its Impact on Training Large Language Modelscited by 59
- 2017Active Learning for Convolutional Neural Networks: A Core-Set Approachcited by 36
- 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
4 more
- 2023Diversity of Thought Improves Reasoning Abilities of LLMscited by 25
- 2023Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Datacited by 19
- 2023AlpaGasus: Training A Better Alpaca with Fewer Datacited by 19
- 2024Influential Language Data Selection via Gradient Trajectory Pursuitcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning
- 2024ConvoSense: Overcoming Monotonous Commonsense Inferences for Conversational AI
- 2023SYNDICOM: Improving Conversational Commonsense with Error-Injection and Natural Language Feedback
- 2022Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts
- 2025Commonsense Generation and Evaluation for Dialogue Systems using Large Language Models
- 2022Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach
- 2023Advancing Transformer's Capabilities in Commonsense Reasoning
- 2023SituatedGen: Incorporating Geographical and Temporal Contexts into Generative Commonsense Reasoning