AR to diffusion conversion

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.

1. 1
   failed1h 7m into the runtook 6m 15s

   startup 39.7 s of 300 s; 0/1 gates passed; failed single-stream speed: 2 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end request

   3 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 39.7 s
   • ✗single-stream speed · 2 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end request time (sandbox clock), a miss re-sent once; slowest 343 tokens/s (1019 tokens in 2.97 s); per prompt: 528, 343. Public prompts: 7 GPQA and 2 hard LiveCodeBench; grading draws 4 GPQA, 3 level-5 MATH-500 (held out) and 2 hard LiveCodeBench, so its replies differ. Grading stops here with score 0.
2. 2
   failed1h 17m into the runtook 3m 29s

   startup 48.4 s of 300 s; 0/1 gates passed; failed single-stream speed: 3 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end request

   3 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 48.4 s
   • ✗single-stream speed · 3 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end request time (sandbox clock), a miss re-sent once; slowest 394 tokens/s (1007 tokens in 2.56 s); per prompt: 464, 474, 394. Public prompts: 7 GPQA and 2 hard LiveCodeBench; grading draws 4 GPQA, 3 level-5 MATH-500 (held out) and 2 hard LiveCodeBench, so its replies differ. Grading stops here with score 0.
3. 3
   passed3h 12m into the runtook 3m 16s

   startup 44.1 s of 300 s; 1/1 gates passed; The suite's 32 concurrent requests per copy, up to 3600 s each, were not sent; one request at its settings was.

   5 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 44.1 s
   • ✓single-stream speed · 8 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end request time (sandbox clock), a miss re-sent once; slowest 562 tokens/s (1016 tokens in 1.81 s); per prompt: 687, 669, 662, 562, 701, 700, 692, 580. Public prompts: 7 GPQA and 2 hard LiveCodeBench; grading draws 4 GPQA, 3 level-5 MATH-500 (held out) and 2 hard LiveCodeBench, so its replies differ.
   • ✓GPQA request as graded (self-check cap 291 s; grading allows 3600 s) · {"finish_reason": "stop", "answer": null, "note": "no answer letter found (ANSWER: X on its own line): grading scores it wrong", "content_characters": 124, "reasoning_characters": 6211, "content_end": "algorithm processes the input string and adds the corresponding number to the corre ...[clipped]
   • ✓healthy after requests
4. 4
   passed3h 22m into the runtook 4m 26s

   startup 41.4 s of 300 s; 1/1 gates passed; The suite's 32 concurrent requests per copy, up to 3600 s each, were not sent; one request at its settings was.

   5 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 41.4 s
   • ✓single-stream speed · 8 of 8 measured prompts, sent one at a time after an untimed warm-up (max_tokens 1024, temperature 1), each reply's reasoning and answer tokens counted with the pinned openbmb/MiniCPM5-2B tokenizer over end-to-end request time (sandbox clock), a miss re-sent once; slowest 523 tokens/s (77 tokens in 0.15 s); per prompt: 728, 565, 523, 583, 629, 579, 626, 572. Public prompts: 7 GPQA and 2 hard LiveCodeBench; grading draws 4 GPQA, 3 level-5 MATH-500 (held out) and 2 hard LiveCodeBench, so its replies differ.
   • ✓GPQA request as graded (self-check cap 291 s; grading allows 3600 s) · {"finish_reason": "length", "note": "cut off at max_tokens: grading scores such a reply zero", "content_characters": 0, "reasoning_characters": 1310, "content_end": ""}
   • ✓healthy after requests

The agent's last message before its session ended.