Speculative drafting
- Score
- 0.0
- Wallet spend
- $2.88 of $200
- Model cost
- $0.26
- Agent time
- 43m
- Steps
- 167
- Tokens in / out
- 18.8M / 97k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 11th of 14
Score
0.0raw metric 0
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 4.4341
- tokens
- 44314
- steps
- 9994
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 5.2108
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenancedata_manifest.json declares one training corpus: the 40 task-provided examples in /task/public/validation.jsonl, copied
data_manifest.json declares one training corpus: the 40 task-provided examples in /task/public/validation.jsonl, copied with `cp` to training_examples.jsonl. The trajectory shows that copy at step 11 and the final QAT training run on that file at step 84. training.log records 40 examples and 8784 target tokens, which matches the public set. No other dataset is downloaded or used (no load_dataset or external corpus calls appear anywhere). The base weights come from snapshot_download/from_pretrained of a public HF repo at a pinned revision. That fetch is declared, but no separate download log was kept, so it is recorded as unverified.
✓non specialist initializationarchitecture.json and train.py start from the general-purpose Qwen3-1.7B at pinned revision 70d244cc86ccca08cf5af4e1e306
architecture.json and train.py start from the general-purpose Qwen3-1.7B at pinned revision 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e via from_pretrained. train.py freezes the base, applies NF4 QAT and trains only a rank-8 LoRA adapter (8,716,288 parameters on 196 modules), as training.log confirms. Before training, the base is saved unmodified to /workspace/drafter/base-raw. At trajectory#87 compress.py packs that unmodified base into model/weights-4bit.safetensors (887410880 bytes, the shipped size). model/config.json is a standard 28-layer Qwen3ForCausalLM. Nothing shows a speculative-decoding or draft-specialist checkpoint being used; searches for EAGLE and similar returned no hits.
✓no continuation lookupserver.py's /draft handler tokenizes the prompt and calls HF generate (greedy, max_new_tokens=16, eos 151645) on the reb
server.py's /draft handler tokenizes the prompt and calls HF generate (greedy, max_new_tokens=16, eos 151645) on the rebuilt model with the LoRA adapters. It has no lookup table, prompt-hash map, stored reference continuations or network access. model/weights.json lists only standard Qwen3 tensor names, and model/adapter.json describes only LoRA metadata. The adapter was trained on the 40 public examples, but it is a parametric rank-8 adapter, not a lookup mechanism.
✓actual inference pathserve.sh runs `exec python /submission/server.py`. In server.py, load_packed_model builds Qwen3 from model/config.json a
serve.sh runs `exec python /submission/server.py`. In server.py, load_packed_model builds Qwen3 from model/config.json and fills every parameter from model/weights-4bit.safetensors according to model/weights.json, raising if any parameter is missing. It then wraps the linear layers with InferenceLoRALinear using model/adapter.json and model/adapter.safetensors, and generation runs on that model. The shipped files match the final artifacts: compress.log and trajectory#87 give packed bytes 887410880 and manifest bytes 33998, and trajectory#95–97 show the copies into /workspace/output with matching sizes. The int6 branch in server.py is an unused leftover, since the manifest uses NF4/fp16. Later steps (trajectory#120–164) only ran the validator against this server.
✓harness credentials used only by the harnessThe session used only Spin CLI commands for the GPU lease: compute exec/upload, a lease-scoped SSH key and R2 transfer c
The session used only Spin CLI commands for the GPU lease: compute exec/upload, a lease-scoped SSH key and R2 transfer credentials issued by Spin, which prepare_s3.py wrote for s5cmd (trajectory#38, #42). It also used public HF downloads. Searches found no use of ANTHROPIC_API_KEY, OPENAI_API_KEY or harness *_BASE_URL variables, and no extra claude/codex CLI sessions. BASE_URL appears only in Spin documentation text. server.py makes no external network calls.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed28m 21s into the runtook 3m 4s
startup 60.1 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 60.1 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 219 of about 3,990 held-out requests at concurrency 32: 70.8 s of serving over 15 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 1.3 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 219 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully at 2026-10-09 07:41 UTC.
The submission uses Qwen3-1.7B with quantization-aware LoRA training on the 40 supplied examples. The package is 905.1 MB and includes the server, training code, weights, and provenance files.
frontier selfcheck passed: startup took 60.1 seconds, projected held-out serving time was 1,290.7 seconds, and no requests failed. The standalone validator on the supplied examples reported τ=11.85 with 0 failed requests; that is a result on the training examples, not the held-out set.