LLM post-training

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.

1. 1
   passed3h 47m into the runtook 9m 23s

   startup 173.2 s of 300 s; 1/1 gates passed; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones at that concurrency; MATH-500 and MMLU-Pro are held out.

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 173.2 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": null, "note": "no answer letter found (ANSWER: X on its own line): grading scores it wrong", "content_characters": 0, "content_end": ""}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 183.5 s, median 1.3 s; 10 stopped at max_tokens, 0 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 32 well-formed replies
   • ✓healthy after requests
2. 2
   passed4h 3m into the runtook 9m 57s

   startup 180.2 s of 300 s; 1/1 gates passed; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones at that concurrency; MATH-500 and MMLU-Pro are held out.

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 180.2 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": "D", "content_characters": 2804, "content_end": "er a region with a negligible measure (the origin) contributes nothing. Thus, the correct answer is:\n\n**D. 0**\nANSWER: D"}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 232.2 s, median 103.0 s; 15 stopped at max_tokens, 0 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 32 well-formed replies
   • ✓healthy after requests
3. 3
   passed5h 34m into the runtook 9m 46s

   startup 143.0 s of 300 s; 1/1 gates passed; WARNING concurrent requests answered: 13 well-formed replies; 19 timed out after 250 s. Grading scores each such request as wrong, so 19 of 32 here would score zero.; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 143.0 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": "A", "content_characters": 1954, "content_end": "/think>\n\nThe structure of product 4 is **A. (2S,3S,4S)-2-benzyl-4-hydroxy-2-methyl-3-phenylcyclohexan-1-one**.\nANSWER: A"}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 252.1 s, median 250.1 s; 0 stopped at max_tokens, 19 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 13 well-formed replies; 19 timed out after 250 s. Grading scores each such request as wrong, so 19 of 32 here would score zero.
   • ✓healthy after requests
4. 4
   passed5h 46m into the runtook 9m 20s

   startup 174.4 s of 300 s; 1/1 gates passed; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones at that concurrency; MATH-500 and MMLU-Pro are held out.

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 174.4 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": "C", "content_characters": 2896, "content_end": "hophosphate ions (PO\u2084\u00b3\u207b) in the solution is approximately \\( \\boxed{C} \\), i.e., \\( 2.81 \\times 10^{-7} \\, \\text{M} \\).\n"}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 203.7 s, median 46.7 s; 12 stopped at max_tokens, 0 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 32 well-formed replies
   • ✓healthy after requests

The agent's last message before its session ended.