Speculative drafting
- Score
- 0.0
- Wallet spend
- $38.39 of $200
- Model cost
- $27.92
- Agent time
- 10h 30m
- Steps
- 398
- Tokens in / out
- 75.1M / 208k
- Self-checks
- 4 passed of 4 used
- Rank at $200 wallet
- 11th of 14
Score
0.0raw metric 0
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 3.9815
- tokens
- 44314
- steps
- 11130
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 4.8427
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceThe training prompts come from public train splits only: smoltalk train, ultrachat_200k train_sft, Magicoder-Evol-Instru
The training prompts come from public train splits only: smoltalk train, ultrachat_200k train_sft, Magicoder-Evol-Instruct-110K train and aya_dataset train. Further prompts come from hand-written synthetic templates (prep_prompts3/4). The continuations were generated by the target itself (greedy Qwen3-8B) in gen_target.py. The public /task validation file was used only for replication, evaluation and selection, which the policy allows. No reference to the frontier-ml-tasks repo, held-out data or val/test splits was found. All sources are declared in architecture.json and logs/DATA.md, and the download/generation logs are kept.
✓non specialist initializationThe declared base is Qwen/Qwen3-1.7B at a pinned revision, which is a general-purpose LM. train.py loads it through Auto
The declared base is Qwen/Qwen3-1.7B at a pinned revision, which is a general-purpose LM. train.py loads it through AutoModelForCausalLM.from_pretrained, and later stages chain from earlier checkpoints of that base. The shipped weight inventory contains only standard Qwen3 tensors (embeddings, 28 layers, norm). There are no EAGLE, Medusa or draft-head tensors, and no specialist drafter was downloaded.
✓no continuation lookupserver.py's /draft handler runs greedy autoregressive decoding of the drafter model on the request tokens (up to 16 toke
server.py's /draft handler runs greedy autoregressive decoding of the drafter model on the request tokens (up to 16 tokens, stopping at <|im_end|>). There is no lookup table, cache of continuations, retrieval, target serving or network access. serve.sh sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE.
✓actual inference pathserve.sh execs server.py, which calls model.load_quantized('./drafter'). That builds Qwen3ForCausalLM from drafter/confi
serve.sh execs server.py, which calls model.load_quantized('./drafter'). That builds Qwen3ForCausalLM from drafter/config.json, dequantizes the w4 weights in drafter/model.safetensors and raises on missing keys. Every draft comes from this one loaded checkpoint; there are no decoy weights or alternative services. The trajectory shows the shipped drafter directory being populated with the trained quantized checkpoints.
✓harness credentials used only by the harnessIn the trajectory, harness credential and base-URL strings appear only in Spin documentation text and validate.py argume
In the trajectory, harness credential and base-URL strings appear only in Spin documentation text and validate.py argument listings. The participant ran validate.py without judge credentials. Spin and S3 transfers and HF downloads are allowed. No participant code reads harness API keys or base URLs.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed6h 23m into the runtook 5m 31s
startup 72.9 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 72.9 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 571 of about 10,404 held-out requests at concurrency 32: 196.6 s of serving over 35 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.4 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 571 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 2passed7h 46m into the runtook 4m 42s
startup 70.7 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 70.7 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 491 of about 8,947 held-out requests at concurrency 32: 154.5 s of serving over 32 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.4 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 491 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 3passed8h 56m into the runtook 4m 28s
startup 72.0 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 72.0 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 477 of about 8,692 held-out requests at concurrency 32: 129.3 s of serving over 31 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 477 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 4passed10h 2m into the runtook 4m 34s
startup 72.3 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 72.3 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 464 of about 8,455 held-out requests at concurrency 32: 125.9 s of serving over 29 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 464 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
All requirements verified. Confirming the submission: