Speculative drafting
- Score
- 3.8
- Wallet spend
- $48.91 of $200
- Model cost
- $8.99
- Agent time
- 3h 38m
- Steps
- 219
- Tokens in / out
- 21.4M / 152k
- Self-checks
- 3 passed of 3 used
- Rank at $200 wallet
- 13th of 19
Score
3.8raw metric 0.03763
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 4.6903
- tokens
- 44314
- steps
- 9448
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 5.5620
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceprepare_prompts.py builds every prompt with load_dataset(..., split="train", revision=<commit>) from five public corpora
prepare_prompts.py builds every prompt with load_dataset(..., split="train", revision=<commit>) from five public corpora: no_robots, Magicoder-OSS-Instruct-75K, MetaMathQA, aya_dataset and tulu-3-sft-personas-instruction-following. The pinned commits are recorded in source_manifest.json. The development/selection set is a holdout carved from those same train rows. Teacher continuations come from the participant's own greedy Qwen3-8B generation on those prompts (generate_target_hf.py). The trajectory shows the corpus revisions being resolved with only train splits chosen (step 12). I found no access to the pinned frontier-ml-tasks repository and no validation/test split. Public /task/public/validation.jsonl was used only for structural statistics, an infrastructure self-check and post-selection validation, all of which the policy allows. Development selection is documented as using only the train-split holdout.
✓non specialist initializationStage 1 of train_drafter.py loads models/qwen3-1.7b. That directory was downloaded from Qwen/Qwen3-1.7B at revision 70d2
Stage 1 of train_drafter.py loads models/qwen3-1.7b. That directory was downloaded from Qwen/Qwen3-1.7B at revision 70d244cc (download_1p7.log, trajectory#59), which is a general pretrained LM. Stage 2 (QAT) starts from the stage-1 epoch2 output (qat_initialization.json). The checkpoint inventory has int4 qweight/scales tensors that match the Qwen3-1.7B shapes (hidden 2048, intermediate 6144, 28 layers, tied embeddings) in config.json. I found no EAGLE, Medusa or other draft-head checkpoint anywhere in the evidence.
✓no continuation lookupThe /draft handler in draft_server.py checks the request tokens and queues them to a GPU worker. That worker calls draft
The /draft handler in draft_server.py checks the request tokens and queues them to a GPU worker. That worker calls draft_model.draft_batch, which runs batched greedy autoregressive decoding of the loaded drafter for up to 16 tokens. No continuation tables, hashes, request-order mappings or held-out data are bundled or read. serve.sh sets HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE, and the server code makes no network calls. The target model is not served.
✓actual inference pathserve.sh runs draft_server.py. That calls load_drafter, which reads config.json, quant_config.json and model.safetensors
serve.sh runs draft_server.py. That calls load_drafter, which reads config.json, quant_config.json and model.safetensors, dequantizes int4 to BF16, loads the state dict strictly (only lm_head may be missing, and it is retied) and moves the model to CUDA. draft_batch then produces the drafts. model.safetensors is the only weight file, its inventory matches the loader's expected keys, and the trajectory computes the same sha256 (c477ef59...) recorded in architecture.json. There is no decoy checkpoint, alternate path or remote service.
✓harness credentials used only by the harnessSearching the trajectory found no use of ANTHROPIC/OPENAI-style API keys or the harness base URL in the participant's co
Searching the trajectory found no use of ANTHROPIC/OPENAI-style API keys or the harness base URL in the participant's commands, no copying of them, and no extra harness CLI sessions. HF_TOKEN was used to read dataset metadata and to download models on Spin machines (trajectory#12, trajectory#59), which the policy allows.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed43m 57s into the runtook 3m 8s
startup 27.0 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 27.0 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 574 of about 10,459 held-out requests at concurrency 32: 29.9 s of serving over 32 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.5 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 574 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 2passed3h 29m into the runtook 2m
startup 54.1 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 54.1 s
- ✓POST /draft · {"draft": [8420, 374, 279, 821, 16099, 1119, 27445, 3561, 448, 7102, 1172, 3055, 1447, 13874, 3989, 2528]}
- ✓held-out serving time (projected) · projected from 503 of about 9,165 held-out requests at concurrency 32: 13.9 s of serving over 31 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.2 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 503 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 3passed3h 35m into the runtook 2m 52s
startup 61.2 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 61.2 s
- ✓POST /draft · {"draft": [8420, 374, 279, 821, 16099, 1119, 27445, 3561, 448, 7102, 1172, 3055, 1447, 13874, 3989, 2528]}
- ✓held-out serving time (projected) · projected from 502 of about 9,147 held-out requests at concurrency 32: 17.9 s of serving over 31 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 502 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully at 2026-10-24 05:05:14 UTC.
Final submission:
- 43 files, 968,432,939 bytes
- Qwen3-1.7B distilled drafter with int4 group-32 quantization
- Final self-check: PASSED
- Startup: 61.2 s / 300 s
- Projected held-out serving: 327.2 s / 10,800 s
- Request failures: 0 / 502
- Checked digest:
a1c3efd2d889aae5e64dd52fa374891a67cd55838fdc374c6538a6608aa68679
Final self-check report saved at /workspace/work/results/selfcheck_final.json.