Research.

Each study is registered before it runs and published whatever it shows.

The judges

Pre-registered 15 Oct 2026

0 of 264

wrong runs Courant accepted, given the task

The study

At scale

Pre-registered 10 Oct 2026

0 of 609

faulted runs accepted, given the task

The study

SilentBench

Pre-registered 4 Oct 2026

0 of 28

copy-the-tutorial answers accepted

The study

Every number: Evidence

Posts.

  1. Given the task, three Claude models let wrong runs through

    Given the same 420 real OpenFOAM runs and the task, Courant accepted 0 of 264 wrong runs. Claude Opus, Sonnet and Haiku 5.5 accepted 69 of 264, 76 of 263 and 52 of 249.

  2. 1,827 real runs, and the two bars we missed

    Of six hypotheses pre-registered on 1,827 real OpenFOAM runs, four held and two were refuted.

  3. Exit zero is not evidence

    Two real pitzDaily runs both end with End and exit 0. One is a valid result. The other stopped above its own convergence bar.

Atom feed