One question per study, frozen when it is answered.
Each study has a permanent page. Once its findings are final and a claim audit has resolved every error it found, the study is frozen: its findings stop changing, and its code, models and datasets are tagged at that point. A number later found wrong is corrected in the repository’s errata. New experiments go into the next study. A study whose own pre-registered stop rule ends it is marked stopped, and its page says why; if it is reopened, its page keeps that record.
- Study001
Frozen 13 Sept 2026 · evidence at study-001
We improved the metric. Then the model stopped answering.
Tool-policy post-training on Qwen3.5-2B: SFT, DPO, and the capability regression the gate missed
Confirmatory call F1 rose from 0.6264 to 0.7470 with SFT. The promoted checkpoint then refused every bare GSM8K question, a regression the promotion gate could not see because its held-out set had no questions that simply asked for an answer.
- Study002
Reopened 25 Sept 2026 · no model result yet
Why the model stopped answering: the mechanism study
The Study 001 refusal/direct-answer regression: its mechanism, its boundary conditions and one corrective intervention
Its first attempt stopped before training by its own pre-registered stop rule: the set of genuinely unanswerable questions could not reach its labeller-agreement floor (0.770 against 0.80). Reopened the same day with a new labelling design fixed in advance: the labellers now agree (0.9525), but only 67 of the questions are ones a model should refuse, under the 385 the safety check needs. Constructed questions about specific future outcomes then brought the safety check to 666 questions (67 natural, 599 constructed). An audit before any training then dropped two planned experiments that could not have tested anything (six runs), cutting the plan from 35 runs to 29. A classifier flags 19,051 training records as likely refusals; a check by two models, not yet run, decides which are corrected, and it must now pass in each source, not only overall. Training is not yet authorised. No GPU time has been spent and no model result exists.