LogicPro: Improving Complex Logical Reasoning via Program-Guided Learning

In this paper, we propose a new data synthesis method called LogicPro, which leverages LeetCode-style algorithm underlineProblems and their corresponding underlineProgram solutions to synthesize Complex underlineLogical Reasoning data in text format. First, we synthesize complex reasoning problems through source algorithm problems and test cases. Then, standard answers and intermediate variable outputs are obtained for each problem based on standard python solutions and test cases. Finally, with the guidance of code intermediate variables, we synthesize the text reasoning process for each reasoning problems. Through this method, we can synthesize data that is difficult, scalable, effective, and comes with golden standard answers and high-quality reasoning processes. As a result, with our 540K synthesized dataset constructed solely from 2,360 algorithm problems, our approach footnoteCode and data are publicly available at https://github.com/jiangjin1999/LogicPro achieves significant improvements in multiple models for the datasets textitBBH27, textitLogicBench, textitDROP, textitAR-LSAT, and textitGSM8K, etc. outperforming a wide range of existing reasoning datasets.