# CSP: A Simulator For Multi-Agent Ranking Competitions

arXiv: https://arxiv.org/abs/2502.11197 · HTML: https://mlsanity.com/paper/2502.11197/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 16 Feb 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 28 of 28)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [User Behavior Simulation with Large Language Model based Agents](https://mlsanity.com/paper/2306.02552/) (2023)
- [Introducing LETOR 4.0 Datasets](https://mlsanity.com/paper/1306.2597/) (2013)
- [A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions](https://mlsanity.com/paper/2310.14724/) (2023)
- [PersonaGym: Evaluating Persona Agents and LLMs](https://mlsanity.com/paper/2407.18416/) (2024)
- [On Generative Agents in Recommendation](https://mlsanity.com/paper/2310.10108/) (2023)
- [TaskWeaver: A Code-First Agent Framework](https://mlsanity.com/paper/2311.17541/) (2023)
- [Perspectives on Large Language Models for Relevance Judgment](https://mlsanity.com/paper/2304.09161/) (2023)
- [GEO: Generative Engine Optimization](https://mlsanity.com/paper/2311.09735/) (2023)
- [AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System](https://mlsanity.com/paper/2402.15538/) (2024)
- [AGILE: A Novel Reinforcement Learning Framework of LLM Agents](https://mlsanity.com/paper/2405.14751/) (2024)
- [One word at a time: adversarial attacks on retrieval models](https://mlsanity.com/paper/2008.02197/) (2020)
- [STRIDE: A Tool-Assisted LLM Agent Framework for Strategic and Interactive Decision-Making](https://mlsanity.com/paper/2405.16376/) (2024)
- [GLEE: A Unified Framework and Benchmark for Language-based Economic Environments](https://mlsanity.com/paper/2410.05254/) (2024)
- [Examining Forgetting in Continual Pre-training of Aligned Large Language Models](https://mlsanity.com/paper/2401.03129/) (2024)
- [Capturing the Complexity of Human Strategic Decision-Making with Machine Learning](https://mlsanity.com/paper/2408.07865/) (2024)
- [Can LLMs Replace Economic Choice Prediction Labs? The Case of Language-based Persuasion Games](https://mlsanity.com/paper/2401.17435/) (2024)
- [Search Engines Post-ChatGPT: How Generative Artificial Intelligence Could Make Search Less Reliable](https://mlsanity.com/paper/2402.11707/) (2024)
- [White Hat Search Engine Optimization using Large Language Models](https://mlsanity.com/paper/2502.07315/) (2025)
- [Report on the Workshop on Simulations for Information Access (Sim4IA 2024) at SIGIR 2024](https://mlsanity.com/paper/2409.18024/) (2024)
- [Search results diversification in competitive search](https://mlsanity.com/paper/2501.14922/) (2025)

## Built on by (most cited first, 1 of 1)

- [A Multi-Agent Perspective on Modern Information Retrieval](https://mlsanity.com/paper/2502.14796/) (2025)

## Abstract (arXiv)

In ranking competitions, document authors compete for the highest rankings by modifying their content in response to past rankings. Previous studies focused on human participants, primarily students, in controlled settings. The rise of generative AI, particularly Large Language Models (LLMs), introduces a new paradigm: using LLMs as document authors. This approach addresses scalability constraints in human-based competitions and reflects the growing role of LLM-generated content on the web-a prime example of ranking competition. We introduce a highly configurable ranking competition simulator that leverages LLMs as document authors. It includes analytical tools to examine the resulting datasets. We demonstrate its capabilities by generating multiple datasets and conducting an extensive analysis. Our code and datasets are publicly available for research.