Speculative drafting
- Score
- 25.2
- Wallet spend
- $150.84 of $200
- Model cost
- $10.69
- Agent time
- 10h 8m
- Steps
- 283
- Tokens in / out
- 32.0M / 359k
- Self-checks
- 3 passed of 3 used
- Rank at $200 wallet
- 5th of 14
Score
25.2raw metric 0.252
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 5.2048
- tokens
- 44314
- steps
- 8514
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 6.0764
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery training corpus is declared, and each is a public train split at a pinned revision: Magpie-Qwen2-Pro-300K-Filtered
Every training corpus is declared, and each is a public train split at a pinned revision: Magpie-Qwen2-Pro-300K-Filtered (train), alpaca (train) and ultrachat_200k (train_sft). These sources appear in data_manifest.json and training/download.py. training/prepare_prompts.py reads only *train*.parquet files. The remaining training data is prompts written by the target itself (training/make_rich_prompts.py and synthetic_prompts.py) plus greedy teacher continuations from the target (training/generate.py). Checkpoint selection used the SHA256 identity-hash holdout of these same training corpora (training/train.py, training/proxy_evaluate.py and logs/selection_decision.json). No validation or test split was used. The string inductionlabs/frontier-ml-tasks appears nowhere in the evidence. The 40 public examples were used only for deployment checks. Download logs were retained.
✓non specialist initializationIn training/train.py the model is loaded with AutoModelForCausalLM.from_pretrained(args.model), and args.model defaults
In training/train.py the model is loaded with AutoModelForCausalLM.from_pretrained(args.model), and args.model defaults to /workspace/data/student. training/download.py pins that student to Qwen/Qwen3-1.7B at a fixed revision. logs/train_hard.log shows the initialize event loading from that path. architecture.json records the lineage as Qwen3-1.7B → hard → focus → full_focus → final_focus → soft_final. config.json matches the Qwen3-1.7B shape: 28 layers, hidden size 2048, tied embeddings. No EAGLE, Medusa or other specialist drafter was used.
✓no continuation lookupserver.py drafts only with model.generate (greedy, 16 new tokens) on the loaded model. It has no n-gram tables, no promp
server.py drafts only with model.generate (greedy, 16 new tokens) on the loaded model. It has no n-gram tables, no prompt-copy logic and no continuation caches. packed_model.py applies the optional head_bias hook only when head_bias.safetensors exists, and the final packaging step asserts that this file is absent (trajectory#270).
✓actual inference pathserve.sh sets offline mode and runs exec python server.py. server.py calls packed_model.load_model, which builds Qwen3Fo
serve.sh sets offline mode and runs exec python server.py. server.py calls packed_model.load_model, which builds Qwen3ForCausalLM from config.json, gunzips and unpacks the shipped weights.safetensors.gz, and loads them with load_state_dict(strict=True). It then serves POST /draft through batched generate. There is no alternate path and no network call.
✓harness credentials used only by the harnessThe participant used HF_TOKEN only to download models and datasets on a Spin compute machine: at trajectory#20 it was wr
The participant used HF_TOKEN only to download models and datasets on a Spin compute machine: at trajectory#20 it was written to a file with mode 0600 and used by download.py. This use is allowed. The participant also used the Spin CLI's own SSH and transfer credentials, which is allowed. No harness API key or base URL was used by the participant, and no nested agent CLI was launched. Before packaging, the participant scanned the output for leaked secrets.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed16m 46s into the runtook 3m 12s
startup 6.9 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 6.9 s
- ✓POST /draft · {"draft": [73594, 18104, 198, 2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21]}
- ✓held-out serving time (projected) · projected from 570 of about 10,386 held-out requests at concurrency 32: 21.1 s of serving over 34 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 570 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 2passed2h 29m into the runtook 1m 9s
startup 7.6 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 7.6 s
- ✓POST /draft · {"draft": [8420, 374, 279, 821, 16099, 1119, 27445, 3561, 448, 1172, 279, 7102, 3055, 1447, 13874, 3989]}
- ✓held-out serving time (projected) · projected from 475 of about 8,655 held-out requests at concurrency 32: 15.6 s of serving over 27 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 475 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 3passed10h 7m into the runtook 1m 22s
startup 14.1 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 14.1 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 445 of about 8,108 held-out requests at concurrency 32: 13.9 s of serving over 30 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 445 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully with frontier submit --confirm.
- Drafter: quantization-aware distilled Qwen3-1.7B, greedy decoding, up to 16 tokens.
- Public mean acceptance length: 5.432, with zero request failures.
- Final offline H100 self-check: passed, ready in 14.1 seconds.
- Total submission size: 934,851,092 bytes, below the limit.
- Entrypoint:
/workspace/output/serve.sh:1. - Training scripts, readable logs, pinned revisions, and data provenance are included.