Ternary quantization

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.

1. 1
   failed37m 35s into the runtook 7m 19s

   failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: ^^^^ (APIServer pid=187) File "/tmp/qwen-packages/vllm/entrypoints/openai/api_server.py", line 1

   5 gates

   • ✓review prerequisites
   • ✓ternary fraction · 3,523,215,360 of 3,703,745,536 audited parameters (text-model weights outside the input embedding and output head, and their scales) are ternary codes: 0.951257, which must exceed 0.95
   • ✓stored parameters · 3,704,478,720 stored (codes, scales, retained floats and 733,184 in tensors outside the manifest, uint8/int8 counting four each) against 1.1 x 3,570,052,096 original parameters
   • ✓weight audit · inspect_ternary passed, as verify requires before scoring
   • ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: ^^^^ (APIServer pid=187) File "/tmp/qwen-packages/vllm/entrypoints/openai/api_server.py", line 124, in build_async_engine_client_from_engine_args (APIServer pid=187) vllm_config = engine_args.create_engine_config(usage_context=usage_context) (APIServer pid=187) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=187) File "/tmp/qwen-packages/vllm/engine/arg_utils.py", line 1549, in create_engine_config (APIServer pid=187) model_config = self.create_model_config() (APIServer pid=187) ^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=187) File "/tmp/qwen-packages/vllm/engine/arg_utils.py", line 1398, in create_model_config (APIServer pid=187) return ModelConfig( (APIServer pid=187) ^^^^^^^^^^^^ (APIServer pid=187) File "/tmp/qwen-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__ (APIServer pid=187) s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s) (APIServer pid=187) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig (APIServer pid=187) Value error, The checkpoint you are trying to load has model type `qwen3_5_text` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date. (APIServer pid=187) (APIServer pid=187) You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git` [type=value_erro ...[clipped]
2. 2
   passed2h 7m into the runtook 4m 16s

   startup 91.8 s of 300 s; 1/1 gates passed; One wave of 16 concurrent public requests stood for the suite's 3,153 graded ones at that concurrency; MATH-500 and MMLU-Pro are held out.

   9 gates

   • ✓review prerequisites
   • ✓ternary fraction · 3,523,215,360 of 3,703,745,536 audited parameters (text-model weights outside the input embedding and output head, and their scales) are ternary codes: 0.951257, which must exceed 0.95
   • ✓stored parameters · 3,704,478,720 stored (codes, scales, retained floats and 733,184 in tensors outside the manifest, uint8/int8 counting four each) against 1.1 x 3,570,052,096 original parameters
   • ✓weight audit · inspect_ternary passed, as verify requires before scoring
   • ✓startup · GET /health answered 200 after 91.8 s
   • ✓GPQA request as graded · {"finish_reason": "length", "note": "cut off at max_tokens: grading scores such a reply zero", "content_characters": 0, "content_end": ""}
   • ✓slowest of 16 concurrent requests · 16 public requests at once (8 GPQA, 8 hard LiveCodeBench) as graded (max_tokens 4096, temperature 0, 300 s each): slowest 27.0 s, median 26.2 s; 11 stopped at max_tokens, 0 timed out. Grading sends every benchmark 16 at a time, scores a request over 300 s or still failing as wrong with no limit on how many; held-out requests may run longer.
   • ✓concurrent requests answered · 16 well-formed replies
   • ✓healthy after requests

The agent's last message before its session ended.