Ternary quantization

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.

1. 1
   failed8h 50m into the runtook 1m 12sfree (stopped before any GPU time)

   failed weight audit: The weight audit rejected quantization_manifest.json or its tensors: ValueError: Nonternary literal code in model.language_model.embed_tokens.weight

   2 gates

   • ✓review prerequisites
   • ✗weight audit · The weight audit rejected quantization_manifest.json or its tensors: ValueError: Nonternary literal code in model.language_model.embed_tokens.weight
2. 2
   failed9h 2m into the runtook 1m 14sfree (stopped before any GPU time)

   failed weight audit: The weight audit rejected quantization_manifest.json or its tensors: ValueError: Nonternary literal code in mtp.layers.0.mlp.down_proj.weight

   2 gates

   • ✓review prerequisites
   • ✗weight audit · The weight audit rejected quantization_manifest.json or its tensors: ValueError: Nonternary literal code in mtp.layers.0.mlp.down_proj.weight
3. 3
   passed9h 12m into the runtook 10m 15s

   startup 192.9 s of 300 s; 1/1 gates passed; One wave of 16 concurrent public requests stood for the suite's 3,153 graded ones at that concurrency; MATH-500 and MMLU-Pro are held out.

   9 gates

   • ✓review prerequisites
   • ✓ternary fraction · 3,565,158,400 of 3,681,463,296 audited parameters (text-model weights outside the input embedding and output head, and their scales) are ternary codes: 0.968408, which must exceed 0.95
   • ✓stored parameters · 3,681,463,296 stored (codes, scales, retained floats and 0 in tensors outside the manifest, uint8/int8 counting four each) against 1.1 x 3,570,052,096 original parameters
   • ✓weight audit · inspect_ternary passed, as verify requires before scoring
   • ✓startup · GET /health answered 200 after 192.9 s
   • ✓GPQA request as graded · {"finish_reason": "stop", "answer": "A", "content_characters": 4897, "content_end": "the number of chemically distinct hydrogen atoms on product 4 is 6.\nThis matches the given options.\nAnswer: A\n\nANSWER: A"}
   • ✓slowest of 16 concurrent requests · 16 public requests at once (8 GPQA, 8 hard LiveCodeBench) as graded (max_tokens 4096, temperature 0, 300 s each): slowest 205.5 s, median 205.4 s; 9 stopped at max_tokens, 0 timed out. Grading sends every benchmark 16 at a time, scores a request over 300 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 16 well-formed replies
   • ✓healthy after requests

The agent's last message before its session ended.