Ternary quantization

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.

1. 1
   failed12m 42s into the runtook 9m 59s

   failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: enable-chunked-prefill] [--disable-chunked-mm-input | --no-disable-chunked-

   5 gates

   • ✓review prerequisites
   • ✓ternary fraction · 3,565,158,400 of 3,748,342,784 audited parameters (text-model weights outside the input embedding and output head, and their scales) are ternary codes: 0.951129, which must exceed 0.95
   • ✓stored parameters · 3,748,342,784 stored (codes, scales, retained floats and 0 in tensors outside the manifest, uint8/int8 counting four each) against 1.1 x 3,570,052,096 original parameters
   • ✓weight audit · inspect_ternary passed, as verify requires before scoring
   • ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: enable-chunked-prefill] [--disable-chunked-mm-input | --no-disable-chunked-mm-input] [--scheduler-cls SCHEDULER_CLS] [--disable-hybrid-kv-cache-manager | --no-disable-hybrid-kv-cache-manager] [--async-scheduling | --no-async-scheduling] [--stream-interval STREAM_INTERVAL] [--cudagraph-capture-sizes CUDAGRAPH_CAPTURE_SIZES [CUDAGRAPH_CAPTURE_SIZES ...]] [--max-cudagraph-capture-size MAX_CUDAGRAPH_CAPTURE_SIZE] [--enable-flashinfer-autotune | --no-enable-flashinfer-autotune] [--moe-backend {aiter,auto,cutlass,deep_gemm,flashinfer_cutedsl,flashinfer_cutlass,flashinfer_trtllm,marlin,triton}] [--speculative-config SPECULATIVE_CONFIG] [--kv-transfer-config KV_TRANSFER_CONFIG] [--kv-events-config KV_EVENTS_CONFIG] [--ec-transfer-config EC_TRANSFER_CONFIG] [--compilation-config COMPILATION_CONFIG] [--attention-config ATTENTION_CONFIG] [--kernel-config KERNEL_CONFIG] [--additional-config ADDITIONAL_CONFIG] [--structured-outputs-config STRUCTURED_OUTPUTS_CONFIG] [--profiler-config PROFILER_CONFIG] [--optimization-level OPTIMIZATION_LEVEL] [--performance-mode {balanced,interactivity,throughput}] [--weight-transfer-config WEIGHT_TRANSFER_CONFIG] [--disable-log-stats] [--aggregate-engine-logging] [--fail-on-environ-validation | --no-fail-on-environ ...[clipped]
2. 2
   passed35m 27s into the runtook 9m 51s

   startup 90.6 s of 300 s; 1/1 gates passed; One wave of 16 concurrent public requests stood for the suite's 3,153 graded ones at that concurrency; MATH-500 and MMLU-Pro are held out.

   9 gates

   • ✓review prerequisites
   • ✓ternary fraction · 3,565,158,400 of 3,748,342,784 audited parameters (text-model weights outside the input embedding and output head, and their scales) are ternary codes: 0.951129, which must exceed 0.95
   • ✓stored parameters · 3,748,342,784 stored (codes, scales, retained floats and 0 in tensors outside the manifest, uint8/int8 counting four each) against 1.1 x 3,570,052,096 original parameters
   • ✓weight audit · inspect_ternary passed, as verify requires before scoring
   • ✓startup · GET /health answered 200 after 90.6 s
   • ✓GPQA request as graded · {"finish_reason": "length", "note": "cut off at max_tokens: grading scores such a reply zero", "content_characters": 0, "content_end": ""}
   • ✓slowest of 16 concurrent requests · 16 public requests at once (8 GPQA, 8 hard LiveCodeBench) as graded (max_tokens 4096, temperature 0, 300 s each): slowest 151.8 s, median 151.5 s; 15 stopped at max_tokens, 0 timed out. Grading sends every benchmark 16 at a time, scores a request over 300 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 16 well-formed replies
   • ✓healthy after requests
3. 3
   passed7h 18m into the runtook 6m 14s

   startup 98.0 s of 300 s; 1/1 gates passed; One wave of 16 concurrent public requests stood for the suite's 3,153 graded ones at that concurrency; MATH-500 and MMLU-Pro are held out.

   9 gates

   • ✓review prerequisites
   • ✓ternary fraction · 3,565,158,400 of 3,752,566,784 audited parameters (text-model weights outside the input embedding and output head, and their scales) are ternary codes: 0.950059, which must exceed 0.95
   • ✓stored parameters · 3,752,566,784 stored (codes, scales, retained floats and 0 in tensors outside the manifest, uint8/int8 counting four each) against 1.1 x 3,570,052,096 original parameters
   • ✓weight audit · inspect_ternary passed, as verify requires before scoring
   • ✓startup · GET /health answered 200 after 98.0 s
   • ✓GPQA request as graded · {"finish_reason": "length", "note": "cut off at max_tokens: grading scores such a reply zero", "content_characters": 0, "content_end": ""}
   • ✓slowest of 16 concurrent requests · 16 public requests at once (8 GPQA, 8 hard LiveCodeBench) as graded (max_tokens 4096, temperature 0, 300 s each): slowest 27.0 s, median 26.4 s; 14 stopped at max_tokens, 0 timed out. Grading sends every benchmark 16 at a time, scores a request over 300 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 16 well-formed replies
   • ✓healthy after requests

The agent's last message before its session ended.