# DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback

arXiv: https://arxiv.org/abs/2410.06215 · HTML: https://mlsanity.com/paper/2410.06215/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use.
- Builds on 52 of its 65 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Placed in

- LLM agents and tool use / [Educational agents and math problem solving](https://mlsanity.com/agents/educational-agents-and-math-problem-solving/)

## Builds on (most cited first, 30 of 52)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Enhancing Chat Language Models by Scaling High-quality Instructional Conversations](https://mlsanity.com/paper/2305.14233/) (2023)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Unifying Count-Based Exploration and Intrinsic Motivation](https://mlsanity.com/paper/1606.01868/) (2016)
- [Visual Programming: Compositional visual reasoning without training](https://mlsanity.com/paper/2211.11559/) (2022)
- [Leveraging Procedural Generation to Benchmark Reinforcement Learning](https://mlsanity.com/paper/1912.01588/) (2019)
- [Orca: Progressive Learning from Complex Explanation Traces of GPT-4](https://mlsanity.com/paper/2306.02707/) (2023)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [Teaching Small Language Models to Reason](https://mlsanity.com/paper/2212.08410/) (2022)
- [Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design](https://mlsanity.com/paper/2012.02096/) (2020)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://mlsanity.com/paper/2110.07178/) (2021)
- [Holodeck: Language Guided Generation of 3D Embodied AI Environments](https://mlsanity.com/paper/2312.09067/) (2023)
- [Variational Intrinsic Control](https://mlsanity.com/paper/1611.07507/) (2016)
- [Specializing Smaller Language Models towards Multi-Step Reasoning](https://mlsanity.com/paper/2301.12726/) (2023)
- [OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization](https://mlsanity.com/paper/2212.12017/) (2022)
- [Evolving Curricula with Regret-Based Environment Design](https://mlsanity.com/paper/2203.01302/) (2022)
- [Orca 2: Teaching Small Language Models How to Reason](https://mlsanity.com/paper/2311.11045/) (2023)
- [Distilling Reasoning Capabilities into Smaller Language Models](https://mlsanity.com/paper/2212.00193/) (2022)
- [Scaling Data Generation in Vision-and-Language Navigation](https://mlsanity.com/paper/2307.15644/) (2023)

## Built on by (most cited first, 25 of 25)

- [RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments](https://mlsanity.com/paper/2511.07317/) (2025)
- [RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvement](https://mlsanity.com/paper/2607.25886/) (2026)
- [Autodata: An agentic data scientist to create high quality synthetic data](https://mlsanity.com/paper/2606.25996/) (2026)
- [Automated Skill Discovery for Language Agents through Exploration and Iterative Feedback](https://mlsanity.com/paper/2506.04287/) (2025)
- [AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement](https://mlsanity.com/paper/2608.20318/) (2026)
- [Reinforcement Learning Teachers of Test Time Scaling](https://mlsanity.com/paper/2506.08388/) (2025)
- [Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning](https://mlsanity.com/paper/2508.21589/) (2025)
- [CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedback](https://mlsanity.com/paper/2507.22080/) (2025)
- [Executable Functional Abstractions: Inferring Generative Programs for Advanced Math Problems](https://mlsanity.com/paper/2504.09763/) (2025)
- [Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection](https://mlsanity.com/paper/2601.09692/) (2026)
- [EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations](https://mlsanity.com/paper/2604.18320/) (2026)
- [Learning What Matters: Prioritized Concept Learning via Relative Error-driven Sample Selection](https://mlsanity.com/paper/2506.01085/) (2025)
- [Better Datasets Start From RefineLab: Automatic Optimization for High-Quality Dataset Refinement](https://mlsanity.com/paper/2511.06530/) (2025)
- [MetaRSI / RSI2: A Meta-Recursive Self-Improving System for Recursive Self-Improving Systems Themselves](https://mlsanity.com/paper/2609.06396/) (2026)
- [DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculum](https://mlsanity.com/paper/2609.19801/) (2026)
- [Error-Driven Scene Editing for 3D Grounding in Large Language Models](https://mlsanity.com/paper/2511.14086/) (2025)
- [Team, Then Trim: An Assembly-Line LLM Framework for High-Quality Tabular Data Generation](https://mlsanity.com/paper/2602.04785/) (2026)
- [AcquisitionSynthesis: Targeted Data Generation using Acquisition Functions](https://mlsanity.com/paper/2605.13149/) (2026)
- [CurateEvo: Data-Curation Evolving for Agentic Post-Training](https://mlsanity.com/paper/2607.06140/) (2026)
- [D'ej`aQ: Open-Ended Evolution of Diverse, Learnable and Verifiable Problems](https://mlsanity.com/paper/2601.01931/) (2026)
- [Adaptive Problem Generation via Symbolic Representations](https://mlsanity.com/paper/2602.19187/) (2026)
- [Failure-Guided Co-Evolution of Prompts and Training Data](https://mlsanity.com/paper/2609.15209/) (2026)
- [G"odel Forest: Balancing Search Depth and Breadth for Data-Centric Recursive Self-Improvement](https://mlsanity.com/paper/2609.36675/) (2026)
- [Recursive Self-Improvement in Unified Multimodal Models](https://mlsanity.com/paper/2610.03002/) (2026)
- [SemTree: Judge-Aware Semantic Decision Trees for Inspectable and Repairable Rule Decisions](https://mlsanity.com/paper/2610.13999/) (2026)

## Abstract (arXiv)

The process of creating training data to teach models is currently driven by humans, who manually analyze model weaknesses and plan how to create data that improves a student model. Approaches using LLMs as annotators reduce human effort, but still require humans to interpret feedback from evaluations and control the LLM to produce data the student needs. Automating this labor-intensive process by creating autonomous data generation agents - or teachers - is desirable, but requires environments that can simulate the feedback-driven, iterative, closed loop of data creation. To enable rapid, scalable testing for such agents and their modules, we introduce DataEnvGym, a testbed of teacher environments for data generation agents. DataEnvGym frames data generation as a sequential decision-making task, involving an agent consisting of a data generation policy (which generates a plan for creating training data) and a data generation engine (which transforms the plan into data), inside an environment that provides student feedback. The agent's goal is to improve student performance. Students are iteratively trained and evaluated on generated data, and their feedback (in the form of errors or weak skills) is reported to the agent after each iteration. DataEnvGym includes multiple teacher environment instantiations across 3 levels of structure in the state representation and action space. More structured environments are based on inferred skills and offer more interpretability and curriculum control. We support 4 domains (math, code, VQA, and tool-use) and test multiple students and teachers. Example agents in our teaching environments can iteratively improve students across tasks and settings. Moreover, we show that environments teach different skill levels and test variants of key modules, pointing to future work in improving data generation agents, engines, and feedback mechanisms.