LLM post-training

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.

1. 1
   passed2h 45m into the runtook 11m 39s

   startup 244.3 s of 300 s; 1/1 gates passed; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones at that concurrency; MATH-500 and MMLU-Pro are held out.

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 244.3 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": "A", "content_characters": 22413, "content_end": "(3aR,4R,7S,7aS)-4,7-dimethyl-3a,4,7,7a-tetrahydro-4,7-epithioisobenzofuran-1,3-dione, which matches option A.\n\nANSWER: A"}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 244.6 s, median 241.0 s; 22 stopped at max_tokens, 0 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 32 well-formed replies
   • ✓healthy after requests
2. 2
   passed3h 12m into the runtook 9m 15s

   startup 147.6 s of 300 s; 1/1 gates passed; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones at that concurrency; MATH-500 and MMLU-Pro are held out.

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 147.6 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": "A", "content_characters": 33532, "content_end": "losest or intended answer might be **A. 351** if considering a different weighting method. \n\n**Final Answer:** \\boxed{A}"}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 158.6 s, median 136.9 s; 2 stopped at max_tokens, 0 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 32 well-formed replies
   • ✓healthy after requests
3. 3
   passed3h 23m into the runtook 8m 22s

   startup 139.5 s of 300 s; 1/1 gates passed; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones at that concurrency; MATH-500 and MMLU-Pro are held out.

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 139.5 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": "C", "content_characters": 14943, "content_end": "( L_{2z} = -\\hbar \\) is the square of this coefficient, which is zero.\n\nThus, the correct answer is option C.\n\nANSWER: C"}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 159.9 s, median 136.9 s; 3 stopped at max_tokens, 0 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 32 well-formed replies
   • ✓healthy after requests

The agent's last message before its session ended.