Synthesizing Tasks for Block-based Programming

Block-based visual programming environments play a critical role in introducing computing concepts to K-12 students. One of the key pedagogical challenges in these environments is in designing new practice tasks for a student that match a desired level of difficulty and exercise specific programming concepts. In this paper, we formalize the problem of synthesizing visual programming tasks. In particular, given a reference visual task rm Tin and its solution code rm Cin, we propose a novel methodology to automatically generate a set (rm Tout, rm Cout) of new tasks along with solution codes such that tasks rm Tin and rm Tout are conceptually similar but visually dissimilar. Our methodology is based on the realization that the mapping from the space of visual tasks to their solution codes is highly discontinuous; hence, directly mutating reference task rm Tin to generate new tasks is futile. Our task synthesis algorithm operates by first mutating code rm Cin to obtain a set of codes rm Cout. Then, the algorithm performs symbolic execution over a code rm Cout to obtain a visual task rm Tout; this step uses the Monte Carlo Tree Search (MCTS) procedure to guide the search in the symbolic tree. We demonstrate the effectiveness of our algorithm through an extensive empirical evaluation and user study on reference tasks taken from the Hour of Code: Classic Maze challenge by Code.org and the Intro to Programming with Karel course by CodeHS.com.