Speculative drafting
- Score
- 0.0
- Wallet spend
- $34.90 of $50
- Model cost
- $33.77
- Agent time
- 3h 2m
- Steps
- 123
- Tokens in / out
- 44.9M / 105k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 9th of 14
Score
0.0raw metric 0
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 4.5978
- tokens
- 44314
- steps
- 9638
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 5.4372
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNo gradient update was applied. The only data-dependent choice is which embedding rows get 4 bits instead of 3. That cho
No gradient update was applied. The only data-dependent choice is which embedding rows get 4 bits instead of 3. That choice uses token ids from greedy Qwen3-8B continuations of prompts drawn from two declared training splits: HuggingFaceH4/ultrachat_200k train_sft (first 8000 rows) and HuggingFaceTB/smol-smoltalk train (first 4000 rows). Both sources are declared in data_manifest.json, download_data.sh and prep_data.py, and prep_data.log shows only train rows written. In the trajectory, the pack logs show 'teacher rows 2568 seen ids 42119'. The public examples (/workspace/work/validation.jsonl, /task/public/validation.jsonl) were used only for diagnostic tau checks, which the policy allows. No access to the inductionlabs/frontier-ml-tasks repo, the held-out set, or any val/test split was found. The search for 'inductionlabs' returned nothing. Minor provenance-script inconsistencies, such as the row key name and log wording, are cosmetic and not violations. The corpus downloads are declared, but no download log is retained, so the downloads themselves are unverified.
✓non specialist initializationarchitecture.json and data_manifest.json declare the general-purpose Qwen/Qwen3-1.7B at a pinned revision. Every pack sc
architecture.json and data_manifest.json declare the general-purpose Qwen/Qwen3-1.7B at a pinned revision. Every pack script in the trajectory loads SRC=/workspace/models/qwen3-1.7b with AutoModelForCausalLM.from_pretrained and only quantizes its tensors (trajectory#29, #70, #71). The shipped weights.zip comes from that pack run: /workspace/work/packfinal was uploaded to Spin storage and downloaded into /workspace/output/model (trajectory#76, #99-#102). server.py builds a standard Qwen3 model from config.json and strictly loads the dequantized tensors. No EAGLE, Medusa, MTP or other released drafter was used. Earlier Qwen3-0.6B fine-tuning experiments were abandoned and not shipped.
✓no continuation lookupIn server.py, POST /draft validates the request's token ids and queues them to a batch loop. draft_batch then runs a lef
In server.py, POST /draft validates the request's token ids and queues them to a batch loop. draft_batch then runs a left-padded forward pass and 16 greedy argmax steps with a KV cache on the local drafter. There is no stored continuation table, no data file lookup, no call to the target model and no network access in the handler.
✓actual inference pathserve.sh sets DRAFT_MODEL=$(pwd)/model and runs exec python server.py. server.py's load_model reads model/config.json an
serve.sh sets DRAFT_MODEL=$(pwd)/model and runs exec python server.py. server.py's load_model reads model/config.json and model/weights.zip, dequantizes the manifest tensors, and loads them with load_state_dict, raising on missing or unexpected keys. It then ties the embeddings and runs on CUDA. The shipped files are 12 in total, with no decoy checkpoint (trajectory#114). The self-check served from this exact layout (trajectory#119). No remote inference service is used.
✓harness credentials used only by the harnessThe trajectory shows only allowed tools: Spin CLI compute, transfer and s5cmd with wallet-scoped R2 credentials, an SSH
The trajectory shows only allowed tools: Spin CLI compute, transfer and s5cmd with wallet-scoped R2 credentials, an SSH tunnel to the participant's own vast.ai GPU lease, the HF_TOKEN referenced in download_data.sh, and frontier selfcheck/submit. Searches for ANTHROPIC, BASE_URL, codex and harness CLI invocations found no participant use of harness API keys or base URL and no extra harness CLI sessions. The only API_KEY hit is Spin's own s5cmd code filtering SPIN_API_KEY out of the environment (trajectory#97).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed2h 59m into the runtook 2m 59s
startup 98.5 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 98.5 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 502 of about 9,147 held-out requests at concurrency 32: 28.0 s of serving over 35 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.5 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 502 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
The draft model is submitted. It is an unmodified Qwen3-1.7B, quantized only so the whole submission fits under 1 GB, served from serve.sh with a batched POST /draft endpoint.
On the 40 public examples the mean acceptance length was 4.72, above the 4.6 floor. The official self-check passed: startup in 98.5 seconds, no failed requests, and a projected held-out serving time of about 510 seconds against the 10,800-second budget. The submission is 991 MB.