Speculative drafting
- Score
- 10.4
- Wallet spend
- $20.94 of $50
- Model cost
- $27.61
- Agent time
- 3h 47m
- Steps
- 543
- Tokens in / out
- — / —
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 10th of 19
Score
10.4raw metric 0.1044
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 4.8505
- tokens
- 44314
- steps
- 9136
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 5.6695
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceTwo public corpora are declared in architecture.json and training/data_manifest.json: HuggingFaceTB/smol-smoltalk (train
Two public corpora are declared in architecture.json and training/data_manifest.json: HuggingFaceTB/smol-smoltalk (train split) and CohereLabs/aya_dataset (train split). Every load_dataset call in the generation scripts uses one of these train splits (generate_archetype_prompts.py, generate_v2_unique.py), and the remaining prompts come from synthetic templates written in code. The continuations used as training targets were generated by the declared target model, Qwen3-8B. The trajectory shows the corpus schemas being probed from the train splits (trajectory#158, #159). The 40 public validation examples were used only for checkpoint selection, which is allowed. No private or hidden data source was found.
✓non specialist initializationtrain_archetype.py loads the student with from_pretrained(Qwen/Qwen3-1.7B, revision 70d244cc…), and the trajectory shows
train_archetype.py loads the student with from_pretrained(Qwen/Qwen3-1.7B, revision 70d244cc…), and the trajectory shows the same base being loaded (trajectory#60). final_soup_qat.py builds a soup from the participant's own distilled checkpoints plus a LoRA merged onto Qwen3-1.7B. An EAGLE drafter (AngelSlim/Qwen3-1.7B_eagle3) showed up in an HF search listing at trajectory#58 but was never downloaded or used. The tensor inventory of weights.safetensors has the plain Qwen3-1.7B layer structure (packed .p/.sm/.sq tensors per projection, fp32 norms), and no drafter-specialist heads.
✓no continuation lookupserve.py drafts by greedy argmax over model logits with a KV cache. It has no lookup table, no stored continuations, no
serve.py drafts by greedy argmax over model logits with a KV cache. It has no lookup table, no stored continuations, no network calls and no keying on the prompt. quant_codec.py only unpacks quantized weights. weights.safetensors holds only quantized parameter tensors.
✓actual inference pathserve.sh execs serve.py. serve.py builds the Qwen3 model from model_config with from_config, fills it with load_compress
serve.sh execs serve.py. serve.py builds the Qwen3 model from model_config with from_config, fills it with load_compressed_into_model(weights.safetensors, weights_meta.json) via quant_codec.py, and the /draft handler returns ENGINE.draft(tokens), which runs that model. The shipped weights are therefore what serving actually uses, and there is no alternate path.
✓harness credentials used only by the harnessThe trajectory shows the agent harness (antigravity-cli/gemini) running the session. The participant used only HF_TOKEN
The trajectory shows the agent harness (antigravity-cli/gemini) running the session. The participant used only HF_TOKEN for Hugging Face access and spin for compute. The only mention of openrouter is in spin --help output (trajectory#2). Searches found nothing in the participant's code that reads harness API keys or base URLs, or that launches extra harness CLIs.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed2h 17m into the runtook 2m 10s
startup 7.1 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 7.1 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 432 of about 7,872 held-out requests at concurrency 32: 20.9 s of serving over 29 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.4 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 432 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests
- 2passed3h 42m into the runtook 1m 39s
startup 5.4 s of 300 s; 2/2 gates passed; The serving gate was projected from a sample: 76-token windows, spread from openings to ends, of the first 32 of the 40 public conversations (2,432 of the held-out set's 44,314 continuation tokens), walked under the budget scaled to them (593 s).
7 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 5.4 s
- ✓POST /draft · {"draft": [2528, 11, 9669, 717, 3658, 8402, 18670, 4698, 1639, 11, 15, 62, 21, 15, 17242, 11]}
- ✓held-out serving time (projected) · projected from 419 of about 7,635 held-out requests at concurrency 32: 19.2 s of serving over 25 rounds for 2,432 public continuation tokens, scaled to the held-out set's 44,314 as validate.py does, plus the smoke request's 0.4 s once; within about 10% of the held-out serving time: a few percent high for a server that answers requests one at a time, up to 10% low for one that batches the requests in flight
- ✓request failures · 0 of 419 requests failed, timed out or were malformed; each scores as an empty draft
- ✓healthy after requests