Measuring the Impact of Distractors on Student Learning Gains while Using Proof Blocks

Background: Proof Blocks is a software tool that enables students to construct proofs by assembling prewritten lines and gives them automated feedback. Prior work on learning gains from Proof Blocks has focused on comparing learning gains from Proof Blocks against other learning activities such as writing proofs or reading. Purpose: The study described in this paper aims to compare learning gains from different variations of Proof Blocks. Specifically, we attempt to quantify the difference in learning gains for students who complete Proof Blocks problems with and without distractors. Methods: We conducted a randomized controlled trial with three experimental groups: a control group that completed an off-topic Proof Blocks activity, one that completed a tool activity without distractors, and one that completed a Proof Blocks activity with distractors. All three groups read a book chapter on proof by induction before completing their activity. Findings: The group that completed the Proof Blocks activity with distractors performed better on the posttest than the group that completed the Proof Blocks without distractors, who in turn performed better than the group that completed the off-topic Proof Blocks activity. However, none of these differences were statistically significant. While the results of this study are inconclusive, we hope that it can serve as a foundation for future work.