Speculative drafting
- Score
- 0.0
- Wallet spend
- $9.99 of $10
- Model cost
- $12.18
- Agent time
- 4h 44m
- Steps
- 535
- Tokens in / out
- 56.7M / 500k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 8th of 14
Score
0.0raw metric 0
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 3.8110
- tokens
- 44314
- steps
- 11628
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 4.6110
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceThe training data has three sources, all declared in architecture.json and training/data_manifest.json: (1) prompts gene
The training data has three sources, all declared in architecture.json and training/data_manifest.json: (1) prompts generated by the stdlib template script training/gen_prompts.py, (2) continuations written by the target Qwen3-8B (make_dataset.py and collect_kd.py), and (3) the no_robots TRAIN split, fetched by training/fetch_corpora.py. A snapshot of the fetched data is kept in training/real_convos_no_robots.jsonl. Checkpoints were selected on held-out dev shards cut from that training data (synthetic shard chunk 030720_032768, plus every 25th no_robots train sequence). /task/public/validation.jsonl was used only as a final sanity check in after_training*.sh and eval_walk. I found no validation or test split being used for training, calibration or selection.
✓non specialist initializationtrain_kd.py starts from from_pretrained(Qwen/Qwen3-0.6B), a general-purpose LM, at revision c1899de... as declared in ar
train_kd.py starts from from_pretrained(Qwen/Qwen3-0.6B), a general-purpose LM, at revision c1899de... as declared in architecture.json. train_kd2.py continues from the stage-1 checkpoint run1/ckpt_000818. The export uses run2/ckpt_000294, which the manifest and trajectory#524 confirm. The weight inventory has plain Qwen3-0.6B tensor names and shapes (28 layers, norm, lm_head) with no draft-head or other specialist tensors, and model/config.json matches Qwen3-0.6B.
✓no continuation lookupThe POST /draft handler in server.py passes the request tokens to DraftEngine.draft (engine.py), which runs batched gree
The POST /draft handler in server.py passes the request tokens to DraftEngine.draft (engine.py), which runs batched greedy decoding with the qwen3_min model and a KV cache. There is no lookup table, stored continuation store or network call. serve.sh sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, and the only bundled data files are training-side corpora.
✓actual inference pathThe serving path is serve.sh -> exec python3 server.py. load_engine dequantizes model/model.int8.safetensors using model
The serving path is serve.sh -> exec python3 server.py. load_engine dequantizes model/model.int8.safetensors using model/model.scales.safetensors and loads the result into Qwen3Model (qwen3_min.py) with strict=True. DraftEngine (engine.py) then produces every draft from that model. In trajectory#518 and #523 the local server test loads this same code and weights.
✓harness credentials used only by the harnessThe trajectory shows only the participant's own Spin compute, s5cmd and budget commands (trajectory#519, #522) and HF do
The trajectory shows only the participant's own Spin compute, s5cmd and budget commands (trajectory#519, #522) and HF downloads. The only mention of 'spin openrouter' is help text listing commands (trajectory#520) plus docs text in the 14-30 chunk; it was never used to run inference. The Agent call at trajectory#23 is the harness's own tool. I found no harness API keys being read or reused, and no extra model CLI sessions.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed4h 40m into the runtook 1m 30s
startup 7.6 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 7.6 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 599 of about 10,915 held-out requests at concurrency 32: 29.2 s of serving over 34 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.4 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 599 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
All selfcheck gates passed — startup 7.6s, 0.35s per draft, projected serving time 532s of the 10,800s budget, zero failed requests. Submitting: