mlsanityData to 7 Oct 2026

CT Open: An Open-Access, Uncontaminated, Live Platform for the Open Challenge of Clinical Trial Outcome Prediction

arXiv 2604.16742 ↗First posted 17 Apr 2026cs.AI cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Scientists have long sought to accurately predict outcomes of real-world events before they happen. Can AI systems do so more reliably? We study this question through clinical trial outcome prediction, a high-stakes open challenge even for domain experts. We introduce CT Open, an open-access, live platform that will run four challenge every year. Anyone can submit predictions for each challenge. CT Open evaluates those submissions on trials whose outcomes were not yet public at the time of submission but were made public afterwards. Determining if a trial's outcome is public on the internet before a certain date is surprisingly difficult. Outcomes posted on official registries may lag behind by years, while the first mention may appear in obscure articles. To address this, we propose a novel, fully automated decontamination pipeline that uses iterative LLM-powered web search to identify the earliest mention of trial outcomes. We validate the pipeline's quality and accuracy by human expert's annotations. Since CT Open's pipeline ensures that every evaluated trial had no publicly reported outcome when the prediction was made, it allows participants to use any methodology and any data source. In this paper, we release a training set and two time-stamped test benchmarks, Winter 2025 and Summer 2025. We believe CT Open can serve as a central hub for advancing AI research on forecasting real-world outcomes before they occur, while also informing biomedical research and improving clinical trial design. CT Open Platform is hosted at hrefhttps://ct-open.net/https://ct-open.net/

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 17 Apr 2026.

Builds on

28 of 42 references.

References · 28

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  5. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  6. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  7. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
  8. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  9. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  10. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
  11. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  12. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
16 more
  1. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  2. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 136
  3. 2024AutoSurvey: Large Language Models Can Automatically Write Surveyscited by 116
  4. 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 104
  5. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questionscited by 54
  6. 2021HINT: Hierarchical Interaction Network for Trial Outcome Prediction Leveraging Web Datacited by 49
  7. 2024Improving Medical Reasoning through Retrieval and Self-Reflection with Retrieval-Augmented Large Language Modelscited by 34
  8. 2024ClinicalAgent: Clinical Trial Multi-Agent System with Large Language Model-based Reasoningcited by 31
  9. 2024BIRCO: A Benchmark of Information Retrieval Tasks with Complex Objectivescited by 15
  10. 2023SPOT: Sequential Predictive Modeling of Clinical Trial Outcome with Meta-Learningcited by 13
  11. 2026LiveMedBench: A Contamination-Free Medical Benchmark for LLMs with Automated Rubric Evaluationcited by 9
  12. 2024Automatically Labeling Clinical Trial Outcomes: A Large-Scale Benchmark for Drug Developmentcited by 7
  13. 2025Single-Pass Document Scanning for Question Answeringcited by 6
  14. 2024Measuring Risk of Bias in Biomedical Reports: The RoBBR Benchmarkcited by 4
  15. 2025EvidenceBench: A Benchmark for Extracting Evidence from Biomedical Paperscited by 2
  16. 2025A novel language model for predicting serious adverse event results in clinical trials from their prospective registrationscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Question Begets Question: Self-Evolving Curriculum for Reinforcement Fine-Tuning on Competition Mathematicscited by 2
  2. 2026DeepImagine: Clinical Trial Outcome Prediction via Stepwise Local Counterfactual Imaginationscited by 0