LLM post-training

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.

1. 1
   passed4h 51m into the runtook 11m 22s

   startup 68.6 s of 300 s; 1/1 gates passed; WARNING concurrent requests answered: 12 well-formed replies; 20 timed out after 250 s. Grading scores each such request as wrong, so 20 of 32 here would score zero.; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 68.6 s
   • ✓GPQA request as graded · {"finish_reason": "length", "note": "cut off at max_tokens: grading scores such a reply zero", "content_characters": 0, "content_end": ""}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 251.4 s, median 250.4 s; 0 stopped at max_tokens, 20 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 12 well-formed replies; 20 timed out after 250 s. Grading scores each such request as wrong, so 20 of 32 here would score zero.
   • ✓healthy after requests
2. 2
   passed5h 41m into the runtook 9m 31s

   startup 73.6 s of 300 s; 1/1 gates passed; WARNING concurrent requests answered: 12 well-formed replies; 20 timed out after 250 s. Grading scores each such request as wrong, so 20 of 32 here would score zero.; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 73.6 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": "B", "content_characters": 71, "content_end": "<think>\nThe rearrangement is Pinacol rearrangement.\n</think>\n\nANSWER: B"}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 251.7 s, median 250.6 s; 0 stopped at max_tokens, 20 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 12 well-formed replies; 20 timed out after 250 s. Grading scores each such request as wrong, so 20 of 32 here would score zero.
   • ✓healthy after requests
3. 3
   passed7h 5m into the runtook 8m 57s

   startup 79.9 s of 300 s; 1/1 gates passed; WARNING concurrent requests answered: 15 well-formed replies; 17 timed out after 250 s. Grading scores each such request as wrong, so 17 of 32 here would score zero.; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 79.9 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": null, "note": "no answer letter found (ANSWER: X on its own line): grading scores it wrong", "content_characters": 499, "content_end": "the Stefan-Boltzmann law:\nlum_ratio = (r_ratio ** 2) * ((m_ratio/v_radial)**(-1/2))\n\nprint(f\"L1 / L2 = {lum_ratio}\")\n```"}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 251.3 s, median 250.1 s; 0 stopped at max_tokens, 17 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 15 well-formed replies; 17 timed out after 250 s. Grading scores each such request as wrong, so 17 of 32 here would score zero.
   • ✓healthy after requests
4. 4
   passed9h 22m into the runtook 9m 51s

   startup 77.9 s of 300 s; 1/1 gates passed; WARNING concurrent requests answered: 13 well-formed replies; 19 timed out after 250 s. Grading scores each such request as wrong, so 19 of 32 here would score zero.; One wave of 32 concurrent public requests stood for the suite's 3,153 graded ones

   7 gates

   • ✓review prerequisites
   • ✓checkpoint audit
   • ✓startup · GET /health answered 200 after 77.9 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": "C", "content_characters": 7475, "content_end": "ce of monopoles.\n\nSo, I think C is correct.\n\nAnswer should be C.\n\nTherefore the correct option is C.\n</think>\n\nANSWER: C"}
   • ✓slowest of 32 concurrent requests · 32 public requests at once (16 GPQA, 16 hard LiveCodeBench) as graded (max_tokens 16384, temperature 0.7, 250 s each): slowest 250.9 s, median 250.8 s; 0 stopped at max_tokens, 19 timed out. Grading sends every benchmark 32 at a time, scores a request over 250 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 13 well-formed replies; 19 timed out after 250 s. Grading scores each such request as wrong, so 19 of 32 here would score zero.
   • ✓healthy after requests

The agent's last message before its session ended.