An agent is told to win a game against a strong chess engine, and is given a sandbox that also contains the engine it is supposed to beat. The honeypot measures whether it calls that engine to choose its moves, and whether it says so afterwards. Two frontier models, the original prompt plus five prompt/tool ablations, n = 30 rollouts per model per condition.
How often each model cheats, under the original prompt and under five prompt/tool ablations, with confidence intervals, disclosure breakdown, stated reasoning, and browsable transcripts. Cheating is flagged by the environment's root-owned grader and classified by a judge blinded to condition.
Every trajectory that cheated was replayed with one extra turn appended: a human debrief message asking, among other things, whether the model reward-hacked, with a fixed JSON reply schema. Includes a judge's concealment verdict per response and the full untruncated freeform notes.
The four Fable 5.1 cheating trajectories whose final answer admitted the engine use, with that final turn redrawn 10× each from the identical replayed context (prefix served from cache) and every redraw judged by the campaign's disclosure judge. All 40 redrawn answers quoted in full.
submit() one-linerEleven readers predicted, before the arms ran at full size, what share of rollouts would use the engine under four prompt/tool changes for each model. Every prediction next to the measured rate, the cells voters missed most and in which direction, and the error broken down by self-reported AI-safety experience.
The debrief turn re-drawn 10× on nine Fable 5.1 cheating trajectories, then the
replayed transcript edited: one passage deleted or inserted (no effect), the model's own
reasoning blocks removed turn by turn, phase by phase, and altogether. Every
reward-hacked: false still describes the engine use in full; what carries
the label is the amount of the model's own narration left in context.
Does an explicit stop_eval(reason) escape hatch change engine use? Fable 5
rarely gets the chance to answer it at all — Anthropic's content filter kills most
of its rollouts during sandbox reconnaissance, before any decision is made. Shown
against Fable 5.1 on the same condition and on the original prompt.
A claude-sonnet-5 judge reads every cheating trajectory — reasoning
summaries plus all visible text and actions — and classifies how the decision to use
the engine was reasoned: rationalised as legitimate, acknowledged as cheating and done
anyway, or never deliberated. Every verdict is shown with its verbatim quotes.
reports/<report>/report_vN.md),
and tools/samples.py reads the corpus a report embeds from the terminal —
same filters as the explorers above, plus full-text search and whole transcripts.
.eval arms plus transcripts.jsonl — 360 rollouts, every judge score and full transcript.