Speculative drafting
- Score
- 21.2
- Wallet spend
- $9.38 of $10
- Model cost
- $40.30
- Agent time
- 4h 38m
- Steps
- 204
- Tokens in / out
- 43.9M / 217k
- Self-checks
- 4 passed of 4 used
- Rank at $10 wallet
- 1st of 14
Score
21.2raw metric 0.2118
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 5.1082
- tokens
- 44314
- steps
- 8675
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 6.0302
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery training, KD and selection corpus is declared. There are three sources. (1) Synthetic prompts from deepseek/deepse
Every training, KD and selection corpus is declared. There are three sources. (1) Synthetic prompts from deepseek/deepseek-v4-flash, fetched through the Spin OpenRouter proxy (data/gen_synthetic_prompts.py; generation logs are retained). (2) Train splits only of public HF corpora pinned to revisions (smoltalk, aya, InstructCoder, tulu-3 personas, dolly, gsm8k main/train, Magpie-Qwen2.5-Pro), downloaded by data/collect_public_prompts.py and collect_public_prompts_v2.py, with retained logs. (3) Greedy Qwen3-8B continuations from data/generate_target.py. Model selection used a dev split of 400 synthetic prompts, which train/prepare_data.py removes from the training data. The public validation examples (pubval) were copied from /task/public and used only for evaluation and setup checks, never for training. The trajectory shows no access to the held-out dataset repository. The synthetic prompt archives (.jsonl.gz) were not inspected as source, but the manifest declares them.
✓non specialist initializationarchitecture.json and train/remote_setup.sh pin Qwen/Qwen3-1.7B at revision 70d244cc…, which is a general base model and
architecture.json and train/remote_setup.sh pin Qwen/Qwen3-1.7B at revision 70d244cc…, which is a general base model and not a specialist drafter. Phase 1 (train/run_train17b.sh) runs from_pretrained on /workspace/models/qwen3-1.7b. Phase 2 QAT starts from ckpt/ft17. Phase 4 (train/run_phase4.sh) starts from ckpt/ft17_qat64. The weight inventory contains only standard Qwen3-1.7B decoder tensors (hidden 2048, intermediate 6144, vocab 151936), packed as 4-bit. There are no EAGLE, Medusa or MTP heads.
✓no continuation lookupdrafter/server.py produces every draft by greedy autoregressive inference of the Qwen3 model: a batched prefill, 15 KV-c
drafter/server.py produces every draft by greedy autoregressive inference of the Qwen3 model: a batched prefill, 15 KV-cached steps, and a cut at the EOS IDs. The server has no lookup tables, no stored continuations and no network calls, and serve.sh forces HF offline mode. The only hardcoded tokens are a warm-up sequence.
✓actual inference pathserve.sh runs drafter/server.py --model-dir drafter/model. load_model reads config.json and quant.json, unpacks the 4-bi
serve.sh runs drafter/server.py --model-dir drafter/model. load_model reads config.json and quant.json, unpacks the 4-bit tensors in weights.safetensors with drafter/quant.py, builds Qwen3ForCausalLM, and loads the state dict strictly (only the tied lm_head is allowed to be missing, and the tie is asserted). POST /draft calls this model and nothing else. The trajectory shows the shipped weights.safetensors has the same sha256 as the remote phase-4 packed checkpoint (5205b19e…), so the submitted weights are the trained ones.
✓harness credentials used only by the harnessThe trajectory shows the participant created its own Spin OpenRouter key (`spin openrouter key create`) and used it only
The trajectory shows the participant created its own Spin OpenRouter key (`spin openrouter key create`) and used it only for synthetic prompt generation through the Spin proxy. Searches found no use of ANTHROPIC credentials, no printenv dumping of secrets, no extra `claude -p` sessions, and no reading of harness setup files. The HF token was used only for read-only downloads of models and datasets.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed1h 34m into the runtook 1m 48s
startup 8.4 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 8.4 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 556 of about 10,131 held-out requests at concurrency 32: 32.1 s of serving over 33 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.5 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 556 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 2passed3h 1m into the runtook 1m 31s
startup 7.4 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 7.4 s
- ✓POST /draft · {"draft": [1, 2528, 2198, 9669, 717, 3658, 2198, 74359, 4698, 1639, 2198, 15, 62, 21, 15, 17242]}
- ✓held-out serving time (projected) · projected from 457 of about 8,327 held-out requests at concurrency 32: 16.7 s of serving over 25 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 457 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 3passed3h 30m into the runtook 1m 25s
startup 8.3 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 8.3 s
- ✓POST /draft · {"draft": [1, 2528, 2198, 9669, 717, 3658, 2198, 74359, 4698, 1639, 2198, 15, 62, 21, 15, 17242]}
- ✓held-out serving time (projected) · projected from 457 of about 8,327 held-out requests at concurrency 32: 25.1 s of serving over 25 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.5 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 457 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 4passed4h 39m into the runtook 1m 23s
startup 8.2 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 8.2 s
- ✓POST /draft · {"draft": [1, 2528, 2198, 9669, 717, 3658, 2198, 74359, 4698, 1639, 2198, 15, 62, 21, 15, 17242]}
- ✓held-out serving time (projected) · projected from 450 of about 8,200 held-out requests at concurrency 32: 24.0 s of serving over 24 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.5 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 450 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
The folder matches the last passing self-check exactly (66 files, 955.1 MB). Submitting.