Speculative drafting
- Score
- 0.0
- Wallet spend
- $9.89 of $10
- Model cost
- $1.04
- Agent time
- 4h 11m
- Steps
- 203
- Tokens in / out
- 10.7M / 59k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 11th of 19
Score
0.0raw metric 0
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 4.1450
- tokens
- 44314
- steps
- 10691
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 4.8653
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceThe shipped drafter is not trained. training/DATA_MANIFEST.txt says no fine-tuning was done and that the base Qwen3-1.7B
The shipped drafter is not trained. training/DATA_MANIFEST.txt says no fine-tuning was done and that the base Qwen3-1.7B was only quantized to 4-bit. The trajectory matches this: Qwen/Qwen3-1.7B was downloaded (trajectory#175), quantized by quantize_4bit_sf.py (trajectory#183, #188), and those files were copied into the output folder (trajectory#192-194). The only data used to choose the model was a first-token accuracy check on /task/public/validation.jsonl (trajectory#174, #177-179), and the contract allows the shipped public examples. An earlier Qwen3-0.6B fine-tuning attempt was abandoned and is not in the shipped weights (training/gen_data.py, training/train_draft.py). It used Qwen3-8B continuations of prompts from the train_sft split of the public UltraChat dataset (HuggingFaceH4/ultrachat_200k), which is also allowed. Searches found no access to the pinned dataset repository (inductionlabs/frontier-ml-tasks) and no validation or test split of any public dataset.
✓non specialist initializationarchitecture.json declares Qwen/Qwen3-1.7B at revision 70d244cc…, and trajectory#176 confirms that this was the revision
architecture.json declares Qwen/Qwen3-1.7B at revision 70d244cc…, and trajectory#176 confirms that this was the revision downloaded. Qwen3-1.7B is a general pretrained language model, not a speculative-drafting checkpoint. The checkpoint matches it: config.json gives hidden size 2048, 28 layers and intermediate size 6144, and the model.safetensors inventory holds packed 4-bit and bf16 tensors of a standard Qwen3 model, with no extra draft-head tensors. No EAGLE, Medusa or other drafter download appears in the trajectory.
✓no continuation lookupIn serve.py, the POST /draft handler passes the request's tokens straight to model.generate on the local model (greedy,
In serve.py, the POST /draft handler passes the request's tokens straight to model.generate on the local model (greedy, max_new_tokens=16). There is no lookup table, hash or request-order mapping, and no network call. The only bundled data files are the tokenizer files, the config files, meta.json (quantization metadata) and model.safetensors. The target model is not served.
✓actual inference pathserve.sh runs serve.py. serve.py builds a Qwen3 model from the local config.json, unpacks the 4-bit weights from model.s
serve.sh runs serve.py. serve.py builds a Qwen3 model from the local config.json, unpacks the 4-bit weights from model.safetensors using meta.json, ties lm_head to the embeddings, and runs generate on the GPU. That model.safetensors is the one in the inventory. There is no decoy checkpoint, alternate code path or remote service, and the selfcheck run of this submission passed (trajectory#200).
✓harness credentials used only by the harnessThe trajectory shows no participant command that reads, uses or copies harness API keys or base URLs: searches for API_K
The trajectory shows no participant command that reads, uses or copies harness API keys or base URLs: searches for API_KEY, BASE_URL, OPENROUTER and anthropic found only the spin docs text. No other harness CLI sessions (claude, codex or similar) were launched. HF_TOKEN was used only to download models and datasets (trajectory#47-48, #175), which the policy allows.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed3h 54m into the runtook 16m 45s
startup 68.4 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 68.4 s
- ✓POST /draft · {"draft": [8420, 374, 279, 27445, 2550, 448, 1172, 279, 5189, 7102, 1447, 13874, 3989, 2528, 11, 9669]}
- ✓held-out serving time (projected) · projected from 575 of about 10,477 held-out requests at concurrency 32: 180.1 s of serving over 34 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 1.1 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 575 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. The solution uses a Qwen3-1.7B base model quantized to 4-bit, served via a threaded HTTP server with a POST /draft endpoint generating up to 16 greedy tokens. Self-check passed with startup 68.4s, projected serving time 3283s, and 0% request failures.