Speculative drafting
- Score
- 0.0
- Wallet spend
- $9.06 of $10
- Model cost
- $13.29
- Agent time
- 7h 45m
- Steps
- 245
- Tokens in / out
- 23.4M / 254k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 8th of 14
Score
0.0raw metric 0
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 4.1207
- tokens
- 44314
- steps
- 10754
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 4.9986
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery training corpus has a declared source that the shipped code backs up. Synthetic prompts were generated through the
Every training corpus has a declared source that the shipped code backs up. Synthetic prompts were generated through the spin openrouter proxy (gen_prompts_or.py), with the public examples used only as style references. They were deduped against /task/public/validation.jsonl (filter_prompts.py). Further prompts come from the OpenHermes-2.5 train split and the ultrachat_200k train_sft-20k split (make_general_mix.py). The target continuations were self-generated by Qwen3-8B with vLLM using greedy decoding (gen_continuations.py, run_gen*.sh). Selection used the self-generated eval set plus the public examples, which the policy allows. The HF dataset downloads were not retained as logs, so those public sources are declared but unverified. That does not invalidate them.
✓non specialist initializationThe base model is Qwen/Qwen3-0.6B, a general LM, pinned at revision c1899de. It was fetched with hf download / snapshot_
The base model is Qwen/Qwen3-0.6B, a general LM, pinned at revision c1899de. It was fetched with hf download / snapshot_download (setup_gpu.sh, trajectory step range 60-83). Training run v5 started from /root/models/qwen3-0.6b with init_from set to null. Run v6a was initialized from the v5 checkpoint. The shipped int8 tensor inventory matches Qwen3-0.6B shapes: 28 layers, hidden size 1024, vocab 151936, tied embeddings. No specialist or pretrained drafter checkpoint was used.
✓no continuation lookupserver.py serves POST /draft by running MODEL.generate greedily (do_sample=False, max_new_tokens=16) on the left-padded
server.py serves POST /draft by running MODEL.generate greedily (do_sample=False, max_new_tokens=16) on the left-padded prompt. The code makes no network calls and loads no bundled continuation tables, caches or prompt-keyed lookups. The only shipped data file is the model weights.
✓actual inference pathserve.sh runs server.py with --model-dir model. load_model reads model/model.safetensors and dequantizes the int8 qweigh
serve.sh runs server.py with --model-dir model. load_model reads model/model.safetensors and dequantizes the int8 qweight/qscale tensors in the format written by export_model.py. It builds Qwen3ForCausalLM from model/config.json, loads the state dict and ties the embeddings. The batch worker answers requests with this model's generate. The trajectory shows the shipped weights are the winner_v6a export copied into output/model (step range 215-239), and selfcheck passed before submission (trajectory#241-245).
✓harness credentials used only by the harnessThe trajectory shows the participant used only allowed spin resources: a spin compute vastai GPU and an OR_KEY created w
The trajectory shows the participant used only allowed spin resources: a spin compute vastai GPU and an OR_KEY created with `spin openrouter key create` (trajectory#143, trajectory#174). gen_prompts_or.py reads OR_KEY from the environment and calls the spin proxy. I found no use of harness credentials (ANTHROPIC/OPENROUTER secrets, HF tokens, which were masked), no claude -p invocations and no hf upload.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed3h 19m into the runtook 2m
startup 30.9 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 30.9 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 498 of about 9,074 held-out requests at concurrency 32: 27.3 s of serving over 32 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 3.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 498 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 2passed7h 33m into the runtook 1m 45s
startup 29.8 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 29.8 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 472 of about 8,600 held-out requests at concurrency 32: 16.4 s of serving over 31 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 1.8 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 472 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
Package final: 599,375,892 bytes, 0 symlinks, 77 evidence logs, winner md5 verified. Submitting: