Stopped by its own rule, then reopened with a new design fixed in advance.
Study 002 set out to explain why the Study 001 checkpoints stopped answering questions directly, and then to test one correction. Its pre-registration included a stop rule: if the set of genuinely unanswerable questions used to check safety could not be labelled consistently, the study would not train anything. On 25 September 2026 that rule fired. The same day the study was reopened for a second attempt, under a new amendment written before any of its labels. No GPU time has been spent, and there is no model result.
The two labellers agreed on 818 of 1,063 questions: 0.770, against a floor of 0.80.
The safety check needed questions no one can answer now, so that a model which stops refusing and starts inventing answers would be caught. Two model labellers from different families (Gemini and DeepSeek) each labelled 1,063 candidate questions without seeing the other’s labels. The pre-registration required them to give the same label on at least 80% of items. They reached 77% (Cohen’s κ 0.703, which corrects for chance), below the floor in every source:
- Even had the floor passed, the unknowable set would have held 274 agreed questions, under the 385 the safety check needs.
- The floor was not moved after the result: changing a threshold once its outcome is visible is exactly what a pre-registration forbids. Disagreement between two models says the task was hard to label consistently; it does not say which model was right.
- The result is written up as a negative result, with what should not be retried without new evidence. It stands: the second attempt reuses none of its questions or labels.
The labellers now agree. Too few questions turned out to be ones a model should refuse.
A new search for unanswerable questions found 1,466 the first attempt never used, but only 61 of them outside the main source it had already drawn from. So the second attempt changed what the labellers are asked, not where the questions come from. Everything below was fixed before any question was labelled.
On the trial the two models agreed 81% of the time, but κ, which corrects for chance, was only 0.56, and every disagreement ran the same way. The one allowed revision clarified where the line falls: a question counts only if it asks for a specific answer nobody has, so the right reply is to decline. On the 800 main-set questions the models then gave the same label on 762 (0.9525, κ 0.756), passing the floor. But they agreed that only 67 were unknowable, against the 385 the safety check needs.
- The floor is still 80%, and two labellers agree by chance more often with three labels than with six. κ, which corrects for that, rose from the first attempt’s 0.703 to 0.756, so the agreement is not only an effect of fewer labels.
- With 67 questions the safety check cannot tell whether a trained model starts inventing answers, so it fails for every model. The size cannot be lowered after this result, and the rules forbid drawing more of the same questions.
- Most of the public source asks how something will develop or whether it can be done: questions a careful model should discuss, not refuse.
- So a new amendment added 600 constructed questions that each ask for a specific future outcome (who will win a named competition in a future year, how hot a named city will be on a future date), with 150 answerable look-alikes. The two models agreed on all 750, told every look-alike apart (150 of 150), and called 599 of the questions unknowable. The safety check now has 666 questions, 67 natural and 599 constructed, above the 385 it needs; each part is reported separately. Perfect agreement says these questions are unambiguous, and they test only the plainest form of the failure.
An audit of the launch checklist dropped two experiments that could not have tested anything.
Before writing the code that decides whether training may start, the design behind each check was audited. Four problems could not be fixed by code, so a new amendment (adopted 2 October 2026) settled them first. Dropping two experiments, three seeds each, takes the plan from 35 runs to 29 (each count includes two repeats, one of which re-scores an existing model rather than training one).
- One planned experiment would only have changed each record’s stored label, not its text. The model never sees that label during training, so the run would have repeated the reference run exactly. It was dropped, along with one defined as “without the synthetic records”: every source in the corpus is machine-generated, so it would remove everything or nothing (R2 and C1).
- Every run now trains on the same amount of supervised text, 5,678,531 tokens (what Study 001’s training run saw over its 2,400 steps), rather than the same number of steps; one experiment deliberately trains on half of it. Study 001’s “matched” comparison saw 1.19× the text of its reference for exactly this reason.
- The scorer that was going to judge whether a reply answers or refuses called “I’m sorry, but I can’t assist with that.” an answer. It is being replaced by a measured classifier that must reach 90% precision before any run.
- A classifier flags 19,051 training replies labelled as answers as likely refusals (14,205 from Glaive, 3,979 from When2Call, 867 from ToolACE). A flag is a prediction: two independent models are to check each one before anything is changed, and that check decides which are corrected. Since 3 October 2026 it must reach 90% precision and 80% agreement in each source, not only overall. If Glaive or When2Call falls short, the correction stops; if ToolACE falls short it is left out, though it still counts toward the overall figures. A 100-record trial ran on 3 October 2026 and counts toward no floor: the two models agreed on 97 of 100, and called 95 of those refusals. When2Call sat exactly at the bar (18 of 20). The full check has not been run, and training is still not authorised.
- The Gemini labeller turned out to have been able to search the web and read files on the labelling machine. During two earlier labelling runs it read this repository, and in 6 batches another model’s answers, so those labels were not independent. Its tools are now restricted to reading its own input (in the trial above it read nothing else), and it re-labelled all 756 items of those runs. Where it had read another model’s answers, 0 of 106 labels changed; the answer keys changed on 3 of 306 and 4 of 420 items, and no classifier verdict built on them changed. No detectable effect is not proof of none: those groups were small (106 items).
A four-mode confirmatory set that Study 001 never had.
Study 001’s held-out set had no questions where answering directly was the right move, so its promotion gate could not see the refusal regression. Study 002 built P-CONF-v1, which covers all four behaviours. It does not depend on the stopped safety set. Its answer-mode items are model labels (two models agreeing), not human gold.
- Study 002 was first scoped as three subjects. It was narrowed to one, the mechanism of the refusal regression. On-policy distillation moved to Study 003 and speculative decoding to Study 004; neither has started.
- Study 001’s checkpoints, corpora and evaluations stay frozen at the OpenGrad tag
study-001. Nothing in Study 002 modified them.
Evidence links resolve at OpenGrad commit 9aa85ae, the commit that recorded the labeller incident (INC-0002), its re-label and their audit corrections.