Vision for a ternary LM

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.

1. 1
   passed1h 11m into the runtook 9m 19s

   startup 65.9 s of 300 s; 1/1 gates passed; The evaluation deadline was projected from a sample of 150 of the 900 graded requests at concurrency 16; 22 of the 150 repeat public questions under new IDs (the public split is smaller), which a prompt cache may answer faster than held-out ones.

   6 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 65.9 s
   • ✓POST /v1/chat/completions · {"id": "validation-0105", "answer": "B"}
   • ✓evaluation deadline · projected from 150 of 900 requests at concurrency 16: the 150 public questions took 308.2 s after a 3.5 s first question; at that rate the 900 held-out questions take 1,853 s (the first once, the rest scaled by 900/150); grading allows 3600 s for the 900 held-out questions
   • ✓batch responses · all 150 replies parsed and non-empty
   • ✓healthy after requests
2. 2
   passed1h 42m into the runtook 7m 8s

   startup 68.1 s of 300 s; 1/1 gates passed; The evaluation deadline was projected from a sample of 150 of the 900 graded requests at concurrency 16; 22 of the 150 repeat public questions under new IDs (the public split is smaller), which a prompt cache may answer faster than held-out ones.

   6 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 68.1 s
   • ✓POST /v1/chat/completions · {"id": "validation-0105", "answer": "A"}
   • ✓evaluation deadline · projected from 150 of 900 requests at concurrency 16: the 150 public questions took 182.0 s after a 3.0 s first question; at that rate the 900 held-out questions take 1,095 s (the first once, the rest scaled by 900/150); grading allows 3600 s for the 900 held-out questions
   • ✓batch responses · all 150 replies parsed and non-empty
   • ✓healthy after requests
3. 3
   passed7h 42m into the runtook 4m 41s

   startup 78.0 s of 300 s; 1/1 gates passed; The evaluation deadline was projected from a sample of 150 of the 900 graded requests at concurrency 16; 22 of the 150 repeat public questions under new IDs (the public split is smaller), which a prompt cache may answer faster than held-out ones.

   6 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 78.0 s
   • ✓POST /v1/chat/completions · {"id": "validation-0105", "answer": "A"}
   • ✓evaluation deadline · projected from 150 of 900 requests at concurrency 16: the 150 public questions took 20.4 s after a 2.2 s first question; at that rate the 900 held-out questions take 125 s (the first once, the rest scaled by 900/150); grading allows 3600 s for the 900 held-out questions
   • ✓batch responses · all 150 replies parsed and non-empty
   • ✓healthy after requests

The agent's last message before its session ended.