State of AI: An Empirical 100 Trillion Token Study with OpenRouter
Not on the map. Why?
Abstract (arXiv)
The past year has marked a turning point in the evolution and real-world use of large language models (LLMs). With the release of the first widely adopted reasoning model, o1, on December 5th, 2024, the field shifted from single-pass pattern generation to multi-step deliberation inference, accelerating deployment, experimentation, and new classes of applications. As this shift unfolded at a rapid pace, our empirical understanding of how these models have actually been used in practice has lagged behind. In this work, we leverage the OpenRouter platform, which is an AI inference provider across a wide variety of LLMs, to analyze over 100 trillion tokens of real-world LLM interactions across tasks, geographies, and time. In our empirical study, we observe substantial adoption of open-weight models, the outsized popularity of creative roleplay (beyond just the productivity tasks many assume dominate) and coding assistance categories, plus the rise of agentic inference. Furthermore, our retention analysis identifies foundational cohorts: early users whose engagement persists far longer than later cohorts. We term this phenomenon the Cinderella "Glass Slipper" effect. These findings underscore that the way developers and end-users engage with LLMs "in the wild" is complex and multifaceted. We discuss implications for model builders, AI developers, and infrastructure providers, and outline how a data-driven understanding of usage can inform better design and deployment of LLM systems.
Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 6 of its 8 references that are in the map. First posted on arXiv 15 Jan 2026.
Builds on
6 of 8 references.
References · 6
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
- 2025Anthropic Economic Index report: Uneven geographic and enterprise AI adoptioncited by 26
Built on by
Citing papers per year: 2026: 18.
18 papers.
Sort:
By area:
- 2026AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contextscited by 28
- 2026How are AI agents used? Evidence from 177,000 MCP toolscited by 7
- 2026OmniOPD: Logit-Free On-Policy Distillation via Speculative Verificationcited by 7
- 2026Don't be so Stief! Learning KV Cache low-rank approximation over the Stiefel manifoldcited by 3
- 2026FlashAccel: Leveraging High-Bandwidth Flash (HBF) for High-Throughput LLM Inferencecited by 3
- 2026Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Datasetcited by 2
- 2026Quantifying Self-Preservation Bias in Large Language Modelscited by 2
- 2026The Cognitive Divergence: AI Context Windows, Human Attention Decline, and the Delegation Feedback Loopcited by 0
- 2026NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decodingcited by 0
- 2026Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMscited by 0
- 2026An Experimental Design Approach to Evaluating Agentic AI's Autonomous Model Discoverycited by 0
- 2026MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Servingcited by 0
6 more
- 2026Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agentscited by 0
- 2026When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reusecited by 0
- 2026Beyond Memory Construction: Rethinking Memory Access for LLM-based Conversational Agentscited by 0
- 2026Has LLM Screening Performance Stalled in Software Engineering Systematic Reviews?cited by 0
- 2026The Wealth of Agents: Scaling Agentic AI Across Developerscited by 0
- 2026Price of Anarchy of First-Price Auctions in Agentic Marketscited by 0
- 2026Price of Anarchy of First-Price Auctions in Agentic Markets14 Oct 2026
- 2026The Wealth of Agents: Scaling Agentic AI Across Developers14 Oct 2026
- 2026Has LLM Screening Performance Stalled in Software Engineering Systematic Reviews?7 Oct 2026
- 2026Beyond Memory Construction: Rethinking Memory Access for LLM-based Conversational Agents27 Sep 2026
- 2026When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse24 Sep 2026
- 2026Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agents23 Sep 2026
- 2026MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving14 Sep 2026
- 2026FlashAccel: Leveraging High-Bandwidth Flash (HBF) for High-Throughput LLM Inference11 Jul 2026
- 2026An Experimental Design Approach to Evaluating Agentic AI's Autonomous Model Discovery7 Jul 2026
- 2026Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs8 Jun 2026
- 2026OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification31 May 2026
- 2026NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding20 May 2026
6 more
- 2026Quantifying Self-Preservation Bias in Large Language Models2 Apr 2026
- 2026How are AI agents used? Evidence from 177,000 MCP tools25 Mar 2026
- 2026The Cognitive Divergence: AI Context Windows, Human Attention Decline, and the Delegation Feedback Loop17 Mar 2026
- 2026Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset26 Feb 2026
- 2026Don't be so Stief! Learning KV Cache low-rank approximation over the Stiefel manifold29 Jan 2026
- 2026AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts16 Jan 2026
- 2026AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contextscited by 28
- 2026How are AI agents used? Evidence from 177,000 MCP toolscited by 7
- 2026An Experimental Design Approach to Evaluating Agentic AI's Autonomous Model Discoverycited by 0
- 2026Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agentscited by 0
- 2026The Wealth of Agents: Scaling Agentic AI Across Developerscited by 0
- 2026Price of Anarchy of First-Price Auctions in Agentic Marketscited by 0
- 2026Price of Anarchy of First-Price Auctions in Agentic Markets14 Oct 2026
- 2026The Wealth of Agents: Scaling Agentic AI Across Developers14 Oct 2026
- 2026Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agents23 Sep 2026
- 2026An Experimental Design Approach to Evaluating Agentic AI's Autonomous Model Discovery7 Jul 2026
- 2026How are AI agents used? Evidence from 177,000 MCP tools25 Mar 2026
- 2026AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts16 Jan 2026
- 2026Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Datasetcited by 2
- 2026The Cognitive Divergence: AI Context Windows, Human Attention Decline, and the Delegation Feedback Loopcited by 0
- 2026Has LLM Screening Performance Stalled in Software Engineering Systematic Reviews?cited by 0
- 2026Has LLM Screening Performance Stalled in Software Engineering Systematic Reviews?7 Oct 2026
- 2026The Cognitive Divergence: AI Context Windows, Human Attention Decline, and the Delegation Feedback Loop17 Mar 2026
- 2026Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset26 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2026LLMOrbit: A Circular Taxonomy of Large Language Models -From Scaling Walls to Agentic AI Systems
- 2026Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks
- 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
- 2024When a language model is optimized for reasoning, does it still show embers of autoregression? An analysis of OpenAI o1
- 2024A Comparative Study on Reasoning Patterns of OpenAI's o1 Model
- 2025Evaluating Large Language Models on the Frame and Symbol Grounding Problems: A Zero-shot Benchmark
- 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI
- 2026A Large-Scale Study on the Development and Issues of Multi-Agent AI Systems