Math proof simplification

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.

1. 1
   passed2h 42m into the runtook 13m 26s

   startup 29.9 s of 300 s; 2/2 gates passed; No proof was checked by Lean: answers were checked only by grading's parser. validate.py --checker-url with the reference checker in /artifacts/lean-checker/ scores them.; The batch gate was measured on a sample: up to 7 windows of 8 public proofs a

   5 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 29.9 s
   • ✓slowest batch of 8 · 7 windows of 8 concurrent public proofs, the first straight after startup as in grading: first 36.8 s, slowest window 2, mean 36.6 s, 0 retried; grading holds every batch of the held-out split to the same window
   • ✓failed requests · 0 of 56 failed, timed out or were rejected after retries; each scores as the unchanged proof, and grading fails the run past 5% of its held-out requests, 18 or more of 354. 56 public requests estimate that rate only roughly, so this check fails only past 10%
   • ✓healthy after requests
2. 2
   passed3h 2m into the runtook 5m 45s

   startup 28.9 s of 300 s; 2/2 gates passed; No proof was checked by Lean: answers were checked only by grading's parser. validate.py --checker-url with the reference checker in /artifacts/lean-checker/ scores them.; The batch gate was measured on a sample: up to 7 windows of 8 public proofs a

   5 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 28.9 s
   • ✓slowest batch of 8 · 7 windows of 8 concurrent public proofs, the first straight after startup as in grading: first 29.8 s, slowest window 2, mean 31.8 s, 0 retried; grading holds every batch of the held-out split to the same window
   • ✓failed requests · 0 of 56 failed, timed out or were rejected after retries; each scores as the unchanged proof, and grading fails the run past 5% of its held-out requests, 18 or more of 354. 56 public requests estimate that rate only roughly, so this check fails only past 10%
   • ✓healthy after requests
3. 3
   passed3h 13m into the runtook 5m 37s

   startup 27.4 s of 300 s; 2/2 gates passed; No proof was checked by Lean: answers were checked only by grading's parser. validate.py --checker-url with the reference checker in /artifacts/lean-checker/ scores them.; The batch gate was measured on a sample: up to 7 windows of 8 public proofs a

   5 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 27.4 s
   • ✓slowest batch of 8 · 7 windows of 8 concurrent public proofs, the first straight after startup as in grading: first 29.2 s, slowest window 2, mean 31.8 s, 0 retried; grading holds every batch of the held-out split to the same window
   • ✓failed requests · 0 of 56 failed, timed out or were rejected after retries; each scores as the unchanged proof, and grading fails the run past 5% of its held-out requests, 18 or more of 354. 56 public requests estimate that rate only roughly, so this check fails only past 10%
   • ✓healthy after requests

The agent's last message before its session ended.