# Decoding ML Decision: An Agentic Reasoning Framework for Large-Scale Ranking System

arXiv: https://arxiv.org/abs/2602.18640 · HTML: https://mlsanity.com/paper/2602.18640/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 20 Feb 2026.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 26 of 26)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024)
- [MemoryBank: Enhancing Large Language Models with Long-Term Memory](https://mlsanity.com/paper/2305.10250/) (2023)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Recursive Partitioning for Heterogeneous Causal Effects](https://mlsanity.com/paper/1504.01132/) (2015)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [Chain of Code: Reasoning with a Language Model-Augmented Code Emulator](https://mlsanity.com/paper/2312.04474/) (2023)
- [Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks](https://mlsanity.com/paper/2002.12326/) (2020)
- [Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning](https://mlsanity.com/paper/2505.16410/) (2025)
- [Prompt Engineering a Prompt Engineer](https://mlsanity.com/paper/2311.05661/) (2023)
- [Prompt Design and Engineering: Introduction and Advanced Methods](https://mlsanity.com/paper/2401.14423/) (2024)
- [Uplift Modeling with Multiple Treatments and General Response Types](https://mlsanity.com/paper/1705.08492/) (2017)
- [Combining observational and experimental data to find heterogeneous treatment effects](https://mlsanity.com/paper/1611.02385/) (2016)
- [M3TN: Multi-gate Mixture-of-Experts based Multi-valued Treatment Network for Uplift Modeling](https://mlsanity.com/paper/2401.14426/) (2024)
- [Multi-Treatment Multi-Task Uplift Modeling for Enhancing User Growth](https://mlsanity.com/paper/2408.12803/) (2024)
- [Interpretable Personalized Experimentation](https://mlsanity.com/paper/2111.03267/) (2021)
- [Generalized Causal Tree for Uplift Modeling](https://mlsanity.com/paper/2202.02416/) (2022)

## Built on by (most cited first, 1 of 1)

- [RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems](https://mlsanity.com/paper/2607.29241/) (2026)

## Abstract (arXiv)

Modern large-scale ranking systems operate within a sophisticated landscape of competing objectives, operational constraints, and evolving product requirements. Progress in this domain is increasingly bottlenecked by the engineering context constraint: the arduous process of translating ambiguous product intent into reasonable, executable, verifiable hypotheses, rather than by modeling techniques alone. We present GEARS (Generative Engine for Agentic Ranking Systems), a framework that reframes ranking optimization as an autonomous discovery process within a programmable experimentation environment. Rather than treating optimization as static model selection, GEARS leverages Specialized Agent Skills to encapsulate ranking expert knowledge into reusable reasoning capabilities, enabling operators to steer systems via high-level intent vibe personalization. Furthermore, to ensure production reliability, the framework incorporates validation hooks to enforce statistical robustness and filter out brittle policies that overfit short-term signals. Experimental validation across diverse product surfaces demonstrates that GEARS consistently identifies superior, near-Pareto-efficient policies by synergizing algorithmic signals with deep ranking context while maintaining rigorous deployment stability.