Speculative drafting
- Score
- 0.0
- Wallet spend
- $41.88 of $200
- Model cost
- $4.21
- Agent time
- 12h
- Steps
- 369
- Tokens in / out
- 33.1M / 79k
- Self-checks
- 0 passed of 0 used
- Rank at $200 wallet
- 14th of 19
Score
0.0raw metric 0
The raw metric (mean acceptance length against Qwen3-8B) is rescaled so that untouched 4-bit Qwen3-1.7B as the drafter (acceptance length 4.6) scores 0 and an acceptance length of 7, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- mean_acceptance_length
- tau
- 3.6778
- tokens
- 44314
- steps
- 12049
- samples
- 200
- draft_tokens
- 16
- tau_macro_over_prompts
- 4.4530
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery training corpus has a declared source in the shipped code and manifest. The prompts come from public train splits:
Every training corpus has a declared source in the shipped code and manifest. The prompts come from public train splits: tatsu-lab/alpaca train, gsm8k main train, mbpp train and SQuAD train. Added to these are hand-written synthetic translation, writing and rewrite prompts, plus the shipped /task/public/validation.jsonl examples. All target continuations were generated with Qwen/Qwen3-8B at a pinned revision, using greedy decoding with enable_thinking=False. The data scripts are evidence/gen_data.py and evidence/gen_data2.py, and the trajectory shows gen_targeted.py (trajectory#308) building the targeted data. The continued fine-tune (trajectory#314) and the later KL distillation (trajectory#347) reused the same /workspace/train_data.jsonl, with Qwen3-8B as teacher. No use of the pinned benchmark dataset repo or of hidden test data was found. The downloads of the public datasets are recorded only through the trajectory calls, which I treat as sufficient though not independently verified.
✓non specialist initializationarchitecture.json declares Qwen/Qwen3-0.6B at revision c1899de…. The trajectory shows that model downloaded and its sha
architecture.json declares Qwen/Qwen3-0.6B at revision c1899de…. The trajectory shows that model downloaded and its sha confirmed (trajectory#27, trajectory#30). evidence/train.py initializes from /workspace/qwen3-0.6b, and train_continue.py (trajectory#314) continues from that trained draft model. model/config.json and the model/model.safetensors inventory match Qwen3-0.6B saved in bnb nf4 format: hidden size 1024, 28 layers, embedding shape 151936x1024. No pre-trained speculative-decoding drafter (EAGLE/Medusa or similar) checkpoint was used.
✓no continuation lookupserver.py tokenizes nothing extra. The POST /draft handler passes the request's token ids to model.generate(max_new_toke
server.py tokenizes nothing extra. The POST /draft handler passes the request's token ids to model.generate(max_new_tokens=16, do_sample=False) and returns the first 16 generated ids. There is no cache of reference continuations, no prompt-matching table, no file lookup and no network call. serve.sh only installs the bundled wheels and runs server.py.
✓actual inference pathserve.sh runs server.py, which loads the tokenizer and an AutoModelForCausalLM from ./model. That directory holds the sh
serve.sh runs server.py, which loads the tokenizer and an AutoModelForCausalLM from ./model. That directory holds the shipped model/model.safetensors (nf4 Qwen3-0.6B) and model/config.json. The trajectory shows the trained model being quantized, bundled, downloaded by md5-verified chunks and moved to /workspace/output/model (trajectory#322 to trajectory#336). Draft tokens come from that model's own greedy generation. There is no alternate serving path.
✓harness credentials used only by the harnessIn the trajectory, HF_TOKEN was only checked for presence and length (trajectory#23) and then used to download the publi
In the trajectory, HF_TOKEN was only checked for presence and length (trajectory#23) and then used to download the public Qwen3 checkpoints (trajectory#24, trajectory#27). Model transfers went through the permitted spin compute upload/download commands (trajectory#336, trajectory#364). Searches found no use of the Anthropic or OpenRouter harness credentials and no hub uploads.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 0 of 4.
The agent never ran a self-check.