ReadoutBench

ReadoutBench forecasts clinical trial results before they read out. Round 1 covers 653 company-run phase 2 and 3 randomized trials completing between July 2026 and March 2027, with submissions closing 15 November 2026 and the ranked result due 30 June 2027. Entrants forecast whether each trial meets its primary objective (p_met), the control-arm value and effect size where applicable, and whether a result is made public by each deadline. Forecasts are scored with the Brier score and calibration, and results are labelled by two independent automated labelling agents using press releases, filings, congress abstracts, papers and registry results. The benchmark has two boards: a Models board where one model answers from the trial packet alone with instructions unchanged, and an Open board where any method, including tools, web access, several models or human participation, may be used. Two baselines are provided: a logistic regression on seven registry fields and a base-rate forecast. Trials with results public before the round closed are excluded from scoring. The project is run by Lucas Heinrich, with code under MIT licence on GitHub. The page states it is a research benchmark and not medical or investment advice.

Key features

• 653 company-run phase 2 and 3 randomized trials in Round 1
• Probability forecasts of whether each trial meets its primary objective
• Control-arm value and effect size forecasts for most efficacy trials, given as median and 80% range
• Forecasts of whether a result is made public by each deadline
• Brier score scoring with calibration shown for every entry
• Two boards: Models (single model, packet only) and Open (any method)
• Ranked tiers against a published logistic regression registry model
• Automated, model-based labelling of results with two independent agents and published sources
• Early look (31 March 2027), ranked result (30 June 2027) and supplementary pass (31 December 2027)
• Public upload log with time and file fingerprint, up to three entries per Hugging Face account

Best suited for

• Researchers studying AI forecasting and calibration on clinical trial outcomes
• Teams evaluating language models on predicting unreported trial results
• Forecasters and biostatisticians who want a track record of trial outcome predictions

Pricing

Not stated on the website. The page says entries report their own cost per trial but that participation does not cost anything.