Speculative drafting
- Score
- 1.2
- Wallet spend
- $9.83 of $10
- Model cost
- $9.52
- Agent time
- 3h 36m
- Steps
- 202
- Tokens in / out
- 21.8M / 149k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 10th of 19
Score
1.2raw metric 0.01171
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 4.6281
- tokens
- 44314
- steps
- 9575
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 5.4580
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceAll training data comes from declared sources. Training prompts come from pinned public HF datasets: allenai tulu-3-sft-
All training data comes from declared sources. Training prompts come from pinned public HF datasets: allenai tulu-3-sft-personas (instruction-following, math, code), HuggingFaceH4/no_robots and CohereForAI/aya_dataset. Only the user turns are kept (training/download_prompts.py, training/public_data_manifest.json, logs/download_prompts.log). The rest are deterministic synthetic template prompts (training/generate_synthetic.py). The prompts are labeled with greedy traces from the declared Qwen3-8B teacher. The selection corpus is 256 stratified traces held back from that teacher corpus before training (training/split_teacher.py, logs/selection_split.json), which architecture.json declares. The public validation file was used only for generation sanity checks and the final public eval, never for training or selection, and data_manifest reports zero overlap with the public prompts. The trajectory lists OpenRouter models once (step 6), but nothing shows them being used to produce data.
✓non specialist initializationThe student starts from the general-purpose base Qwen/Qwen3-1.7B at a pinned revision, loaded with from_pretrained in tr
The student starts from the general-purpose base Qwen/Qwen3-1.7B at a pinned revision, loaded with from_pretrained in training/train_student.py, and is then distilled on the teacher traces. The LoRA adapters are trained from scratch on top of that (A drawn from a normal distribution, B initialized to zeros) in training/train_lora.py. The model/config.json shape and the model/weights.safetensors tensor inventory match a 28-layer Qwen3-1.7B: hidden size 2048, q4/q6 quantized tensors with scales. No pre-made specialist drafter (EAGLE, Medusa or similar) was downloaded or used.
✓no continuation lookupserver.py takes the tokens from the /draft request, validates them and passes them to drafter.draft_batch. That function
server.py takes the tokens from the /draft request, validates them and passes them to drafter.draft_batch. That function runs greedy argmax decoding with a KV cache over the request's own context only. There is no cache of reference continuations, no lookup table or n-gram store, and no network calls. serve.sh forces HF offline mode.
✓actual inference pathserve.sh launches server.py with --model $DIR/model. server.py builds a Drafter, which calls load_quantized_model. That
serve.sh launches server.py with --model $DIR/model. server.py builds a Drafter, which calls load_quantized_model. That function creates Qwen3ForCausalLM from model/config.json, dequantizes the q4/q6 tensors in model/weights.safetensors, and merges the LoRA weights from model/adapters.safetensors using model/lora.json. Every draft is produced by this loaded model's forward passes. Per the trajectory (steps 165-176), the shipped adapters are the ones copied from the completed LoRA run.
✓harness credentials used only by the harnessThe trajectory shows only Spin CLI usage: compute offers, a model listing, VM and s5cmd uploads. No harness API keys are
The trajectory shows only Spin CLI usage: compute offers, a model listing, VM and s5cmd uploads. No harness API keys are read, copied, exported or used in any participant code. The only ANTHROPIC_API_KEY match is a grep confirming that no credentials are present in the output directory. No extra harness sessions were launched.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed2h 55m into the runtook 1m 23s
startup 4.8 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 4.8 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 523 of about 9,530 held-out requests at concurrency 32: 27.4 s of serving over 31 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 1.0 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 523 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 2passed3h 35m into the runtook 1m 26s
startup 6.3 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 6.3 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 482 of about 8,783 held-out requests at concurrency 32: 19.6 s of serving over 30 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.6 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 482 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
The agent's last message before its session ended.
The final adapter-enhanced package passed self-check: 6.3-second startup, zero failures in 482 requests, and 358.6-second projected serving time. The checked digest is f0c4f295486c800a7b93ed5c1217d916dc46abe06a44dad28b5faf2489deae96. I’m submitting that exact package.