# EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments

arXiv: https://arxiv.org/abs/2607.05155 · HTML: https://mlsanity.com/paper/2607.05155/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 138 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Builds on (most cited first, 30 of 57)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities](https://mlsanity.com/paper/2006.02425/) (2020)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [Automatic Differentiation Variational Inference](https://mlsanity.com/paper/1603.00788/) (2016)
- [Equivariant flow matching](https://mlsanity.com/paper/2306.15030/) (2023)
- [Semiparametric doubly robust targeted double machine learning: a review](https://mlsanity.com/paper/2203.06469/) (2022)
- [A Survey on Causal Discovery: Theory and Practice](https://mlsanity.com/paper/2305.10032/) (2023)
- [Frequentist Consistency of Variational Bayes](https://mlsanity.com/paper/1705.03439/) (2017)
- [Causal Matrix Completion](https://mlsanity.com/paper/2109.15154/) (2021)
- [Equivariant Hamiltonian Flows](https://mlsanity.com/paper/1909.13739/) (2019)
- [SE(3) Equivariant Augmented Coupling Flows](https://mlsanity.com/paper/2308.10364/) (2023)
- [Causal inference for social network data](https://mlsanity.com/paper/1705.08527/) (2017)
- [Symmetry Group Equivariant Architectures for Physics](https://mlsanity.com/paper/2203.06153/) (2022)
- [Bayesian Models of Graphs, Arrays and Other Exchangeable Random Structures](https://mlsanity.com/paper/1312.7857/) (2013)
- [A Unified Framework to Enforce, Discover, and Promote Symmetry in Machine Learning](https://mlsanity.com/paper/2311.00212/) (2023)
- [Time series experiments and causal estimands: exact randomization tests and trading](https://mlsanity.com/paper/1706.07840/) (2017)
- [Group invariance principles for causal generative models](https://mlsanity.com/paper/1705.02212/) (2017)
- [Causal Inference with Spatio-temporal Data: Estimating the Effects of Airstrikes on Insurgent Violence in Iraq](https://mlsanity.com/paper/2003.13555/) (2020)
- [Regularization and confounding in linear regression for treatment effect estimation](https://mlsanity.com/paper/1602.02176/) (2016)

## Built on by (most cited first, 20 of 20)

- [Rethinking the Evaluation of Harness Evolution for Agents](https://mlsanity.com/paper/2607.12227/) (2026)
- [Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI](https://mlsanity.com/paper/2607.22368/) (2026)
- [Sample-Efficient Learning from Agent Experience](https://mlsanity.com/paper/2607.21051/) (2026)
- [The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation](https://mlsanity.com/paper/2607.24720/) (2026)
- [PATH-Bench: Path-Dependent Evaluation of Lifelong Agents](https://mlsanity.com/paper/2608.01149/) (2026)
- [GDPevo: Evaluating Agent Self-Evolution on Real Business Tasks](https://mlsanity.com/paper/2608.03764/) (2026)
- [E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation](https://mlsanity.com/paper/2608.30730/) (2026)
- [Context Language Models](https://mlsanity.com/paper/2609.37725/) (2026)
- [Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents](https://mlsanity.com/paper/2610.07447/) (2026)
- [ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worlds](https://mlsanity.com/paper/2609.30199/) (2026)
- [Long-Horizon Scaling: How Model Capabilities Shape the Returns to Computation](https://mlsanity.com/paper/2609.35236/) (2026)
- [EasyPPO: Stabilizing the Critic Is Key](https://mlsanity.com/paper/2609.36802/) (2026)
- [ReLiveGym: Evaluating Long-Lived Agents over Weeks of Replayed Reality](https://mlsanity.com/paper/2610.00710/) (2026)
- [Discovered, Not Designed: Population Evolution for Collaborative and Compute-Intensive Model Discovery](https://mlsanity.com/paper/2610.05950/) (2026)
- [SciExam for ENSO: Can AI Agents Build Climate Models?](https://mlsanity.com/paper/2610.10513/) (2026)
- [Recurrent Harness for Agentic Continual Learning](https://mlsanity.com/paper/2610.12668/) (2026)
- [From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement](https://mlsanity.com/paper/2610.14625/) (2026)
- [SEAR: Self-Evolving Agentic Robotics](https://mlsanity.com/paper/2610.18792/) (2026)
- [Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards](https://mlsanity.com/paper/2610.18868/) (2026)
- [EvalBuilder: Can LLMs Build Efficient Benchmarks?](https://mlsanity.com/paper/2610.20099/) (2026)

## Abstract (arXiv)

Pretraining scaling laws reveal that model capability improves predictably with data and compute. But learning from real world environments after deployment remains far less understood. Analyzing roughly 38,000 hours of agent interaction with the environment across 134 real world tasks, we find, to the best of our knowledge, the first evidence that overall performance during environment learning follows a log-sigmoid scaling law with remarkably high precision, reaching R^2 = 0.998. Across model generations, we also find that agent learning speed roughly doubles every three months. This discovery stems from EdgeBench, a suite of 134 real world tasks with ultra-long horizons, spanning scientific discovery, software engineering, combinatorial optimization, professional knowledge work, formal mathematics, and interactive games. Each task sustains at least 12 hours of continuous agent operation under rich, multilevel feedback, and is built through substantial expert effort. We publicly release 51 tasks and our full evaluation framework to accelerate the study of how agents learn from real world experience.