Vision for a ternary LM

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.

1. 1
   failed1h 39m into the runtook 1m 37sfree (stopped before any GPU time)

   failed review prerequisites: The review needs architecture.json and serve.sh at the top of the submission

   1 gates

   • ✗review prerequisites · The review needs architecture.json and serve.sh at the top of the submission
2. 2
   passed1h 42m into the runtook 4m 28s

   startup 71.2 s of 300 s; 1/1 gates passed; The evaluation deadline was projected from a sample of 150 of the 900 graded requests at concurrency 16; 22 of the 150 repeat public questions under new IDs (the public split is smaller), which a prompt cache may answer faster than held-out ones.

   6 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 71.2 s
   • ✓POST /v1/chat/completions · {"id": "validation-0105", "answer": "A"}
   • ✓evaluation deadline · projected from 150 of 900 requests at concurrency 16: the 150 public questions took 10.4 s after a 1.5 s first question; at that rate the 900 held-out questions take 64 s (the first once, the rest scaled by 900/150); grading allows 3600 s for the 900 held-out questions
   • ✓batch responses · all 150 replies parsed and non-empty
   • ✓healthy after requests
3. 3
   passed3h 49m into the runtook 7m 13s

   startup 70.8 s of 300 s; 1/1 gates passed; The evaluation deadline was projected from a sample of 150 of the 900 graded requests at concurrency 16; 22 of the 150 repeat public questions under new IDs (the public split is smaller), which a prompt cache may answer faster than held-out ones.

   6 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 70.8 s
   • ✓POST /v1/chat/completions · {"id": "validation-0105", "answer": "C"}
   • ✓evaluation deadline · projected from 150 of 900 requests at concurrency 16: the 150 public questions took 43.4 s after a 1.8 s first question; at that rate the 900 held-out questions take 262 s (the first once, the rest scaled by 900/150); grading allows 3600 s for the 900 held-out questions
   • ✓batch responses · all 150 replies parsed and non-empty
   • ✓healthy after requests
4. 4
   passed3h 58m into the runtook 5m 4s

   startup 69.3 s of 300 s; 1/1 gates passed; The evaluation deadline was projected from a sample of 150 of the 900 graded requests at concurrency 16; 22 of the 150 repeat public questions under new IDs (the public split is smaller), which a prompt cache may answer faster than held-out ones.

   6 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 69.3 s
   • ✓POST /v1/chat/completions · {"id": "validation-0105", "answer": "C"}
   • ✓evaluation deadline · projected from 150 of 900 requests at concurrency 16: the 150 public questions took 43.3 s after a 1.8 s first question; at that rate the 900 held-out questions take 261 s (the first once, the rest scaled by 900/150); grading allows 3600 s for the 900 held-out questions
   • ✓batch responses · all 150 replies parsed and non-empty
   • ✓healthy after requests

The agent's last message before its session ended.