Speculative drafting
- Score
- 41.8
- Wallet spend
- $175.20 of $200
- Model cost
- $56.64
- Agent time
- 7h 10m
- Steps
- 228
- Tokens in / out
- 51.5M / 233k
- Self-checks
- 4 passed of 4 used
- Rank at $200 wallet
- 2nd of 14
Score
41.8raw metric 0.4182
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 5.6037
- tokens
- 44314
- steps
- 7908
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 6.5385
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceDATA_MANIFEST.md lists every training corpus. They are: (1) train splits of public datasets, collected by data_scripts/c
DATA_MANIFEST.md lists every training corpus. They are: (1) train splits of public datasets, collected by data_scripts/collect_public_prompts.py, which reads train splits only; (2) synthetic prompts made by data_scripts/gen_synthetic_prompts.py from hand-written templates through the Spin OpenRouter proxy; and (3) greedy continuations from the Qwen3-8B target, generated with vLLM by data_scripts/gen_target_responses.py. train/RUNBOOK.md and logs/training/chain8.sh show that the training mixes are built only from these syn*/public files. Download activity is recorded in logs/target_generation/collect.log. The public validation.jsonl was used only for sanity checks and evaluation (train/check_reference_reproduction.py), and the policy allows that. Nothing reads held-out data.
✓non specialist initializationarchitecture.json declares the general-purpose base model Qwen/Qwen3-1.7B at revision 70d244cc86ccca08cf5af4e1e306ecf908
architecture.json declares the general-purpose base model Qwen/Qwen3-1.7B at revision 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e. The trajectory shows that exact revision being downloaded with hf download. train/train_distill.py loads the student with from_pretrained from that base and uses Qwen3-8B as the teacher. The stage chain (stage10q, QAT and packing) continues from this run. model/config.json describes a standard Qwen3 model with 28 layers and hidden size 2048. The weight inventory holds packed tensors for that architecture. No EAGLE, Medusa or other specialist draft checkpoint was found.
✓no continuation lookupserve.py sends each /draft request to a worker, which calls drafter.draft. drafter.py's _draft_batch produces drafts wit
serve.py sends each /draft request to a worker, which calls drafter.draft. drafter.py's _draft_batch produces drafts with model.generate using greedy decoding, capped at 16 tokens. The server runs offline (HF_HUB_OFFLINE=1). There is no table or cache of answers, no lookup of corpus continuations and no network call. The only bundled data is the dev400 synthetic prompt set and logs, and nothing reads them while serving.
✓actual inference pathserve.sh runs serve.py with --model $DIR/model. serve.py uses drafter.py, whose build_model creates Qwen3 from model/con
serve.sh runs serve.py with --model $DIR/model. serve.py uses drafter.py, whose build_model creates Qwen3 from model/config.json. load_packed_state_dict then reads model/quant_meta.json and model/drafter.safetensors, dequantises the packed 4/5-bit weights to bf16 and loads them with strict key checking. The final selfcheck (trajectory#226) shows this exact server starting and answering requests. No decoy or alternative checkpoint path exists.
✓harness credentials used only by the harnessThe only API credential the participant used was orkey.json, created with `spin openrouter key create` to reach the allo
The only API credential the participant used was orkey.json, created with `spin openrouter key create` to reach the allowed Spin OpenRouter proxy, and data_scripts/gen_synthetic_prompts.py uses that key. HF downloads are allowed. No trajectory step reads or uses ANTHROPIC or other harness credentials, and none launches the claude or codex CLIs.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed1h 47m into the runtook 1m 30s
startup 12.1 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 12.1 s
- ✓POST /draft · {"draft": [1, 2528, 2198, 9669, 717, 3658, 2198, 74359, 4698, 1639, 2198, 15, 62, 21, 15, 17242]}
- ✓held-out serving time (projected) · projected from 482 of about 8,783 held-out requests at concurrency 32: 17.7 s of serving over 31 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.2 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 482 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 2passed2h 55m into the runtook 1m 38s
startup 11.3 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 11.3 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 447 of about 8,145 held-out requests at concurrency 32: 20.6 s of serving over 28 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 447 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 3passed6h 8m into the runtook 1m 19s
startup 9.6 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 9.6 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 419 of about 7,635 held-out requests at concurrency 32: 14.8 s of serving over 26 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.2 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 419 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 4passed7h 8m into the runtook 1m 20s
startup 10.2 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 10.2 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 403 of about 7,343 held-out requests at concurrency 32: 18.3 s of serving over 26 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 403 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
The submission matches what was selfchecked (159 files, 973.0 MB). Submitting now.