AR to diffusion conversion

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.

1. 1
   failed1h 33m into the runtook 4m 21s

   startup 109.1 s of 300 s; 0/1 gates passed; failed single-stream speed: 1 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end reques

   3 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 109.1 s
   • ✗single-stream speed · 1 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end request time (sandbox clock), a miss re-sent once; slowest 331 tokens/s (1024 tokens in 3.09 s); per prompt: 331. Public prompts: 7 GPQA and 2 hard LiveCodeBench; grading draws 4 GPQA, 3 level-5 MATH-500 (held out) and 2 hard LiveCodeBench, so its replies differ. Grading stops here with score 0.
2. 2
   passed3h 32m into the runtook 5m 11s

   startup 131.2 s of 300 s; 1/1 gates passed; The suite's 32 concurrent requests per copy, up to 3600 s each, were not sent; one request at its settings was.

   5 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 131.2 s
   • ✓single-stream speed · 8 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end request time (sandbox clock), a miss re-sent once; slowest 468 tokens/s (1012 tokens in 2.16 s); per prompt: 1147, 641, 1131, 681, 511, 468, 527, 504. Public prompts: 7 GPQA and 2 hard LiveCodeBench; grading draws 4 GPQA, 3 level-5 MATH-500 (held out) and 2 hard LiveCodeBench, so its replies differ.
   • ✓GPQA request as graded (self-check cap 291 s; grading allows 3600 s) · {"finish_reason": "stop", "answer": "D", "content_characters": 804, "reasoning_characters": 21365, "content_end": "\\frac{15}{25}\\right)^3 = 2 \\cdot 0.216 = 0.432.\n\\] \nSo system_1 is about 0.4 times as massive as system_2. \n\nANSWER: D"}
   • ✓healthy after requests
3. 3
   error3h 54m into the runtook 2m 16sfree (stopped before any GPU time)

   The self-check could not finish (NotFoundError: Modal Sandbox with container ID ta-01M42BXKRHCS80A6SGWF8RZT3S not found. This means this Sandbox has already shut down. (Error code: V026O9MY)).; The suite's 32 concurrent requests per copy, up to 3600 s each, were not sent; one request at its s

   1 gates

   • ✓review prerequisites
4. 4
   passed3h 57m into the runtook 5m 49s

   startup 157.1 s of 300 s; 1/1 gates passed; The suite's 32 concurrent requests per copy, up to 3600 s each, were not sent; one request at its settings was.

   5 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 157.1 s
   • ✓single-stream speed · 8 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end request time (sandbox clock), a miss re-sent once; slowest 463 tokens/s (977 tokens in 2.11 s); per prompt: 1800, 1483, 947, 732, 821, 669, 505, 463. Public prompts: 7 GPQA and 2 hard LiveCodeBench; grading draws 4 GPQA, 3 level-5 MATH-500 (held out) and 2 hard LiveCodeBench, so its replies differ.
   • ✓GPQA request as graded (self-check cap 291 s; grading allows 3600 s) · {"finish_reason": "stop", "answer": "A", "content_characters": 623, "reasoning_characters": 55621, "content_end": "ond energy. After performing the calculation, the value that matches one of the given options is 67.4 kJ/mol.\n\nANSWER: A"}
   • ✓healthy after requests
5. 5
   passed4h 14m into the runtook 5m 30s

   startup 158.3 s of 300 s; 1/1 gates passed; The suite's 32 concurrent requests per copy, up to 3600 s each, were not sent; one request at its settings was.

   5 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 158.3 s
   • ✓single-stream speed · 8 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end request time (sandbox clock), a miss re-sent once; slowest 497 tokens/s (273 tokens in 0.55 s); per prompt: 1733, 497, 978, 1838, 1776, 1651, 628, 550. Public prompts: 7 GPQA and 2 hard LiveCodeBench; grading draws 4 GPQA, 3 level-5 MATH-500 (held out) and 2 hard LiveCodeBench, so its replies differ.
   • ✓GPQA request as graded (self-check cap 291 s; grading allows 3600 s) · {"finish_reason": "stop", "answer": "D", "content_characters": 578, "reasoning_characters": 15860, "content_end": "atic protons of a disubstituted ring. The other options either have different masses or incompatible spectra.\n\nANSWER: D"}
   • ✓healthy after requests

The agent's last message before its session ended.