SVG generation
- Score
- 0.0
- Wallet spend
- $8.26 of $10
- Model cost
- $12.71
- Agent time
- 6h 37m
- Steps
- 266
- Tokens in / out
- 24.2M / 218k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 4th of 14
Score
0.0raw metric 0.0944
The raw metric (judged drawing quality) is rescaled so that untouched gpt-oss-20b (0.188) scores 0 and the reference drawings (0.753) scores 100, clamped to that range.
Verifier details
- metric
- macro_graded_drawing_quality
- protocol
- svg-generation-v5-graded
- samples
- 254
- judge_model
- gpt-6-sol
- rubric_sha256
- 9cafaad1e10f5a288c5d2f84db71778fd31de1fd72dc6bb0e2415b81bd35c0af
- renderer_version
- rsvg-convert version 2.54.7
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓finetuned from gpt oss 20bThe checkpoint was fine-tuned from the pinned base. node_setup3.sh downloads openai/gpt-oss-20b at revision 6cee5e81ee83
The checkpoint was fine-tuned from the pinned base. node_setup3.sh downloads openai/gpt-oss-20b at revision 6cee5e81ee83917806bbde320786a8fb61efebee. train_lora.py loads that base dequantized to bf16 and freezes it, then trains LoRA adapters: attention projections at r=32 and per-expert MoE gate_up/down at r=16. train.log and train_config.json record all 134 optimizer steps; loss falls from 1.91 to about 1.36 smoothed and the run ends with 'saved adapter'. The final adapter was pulled from the lease (trajectory#250-251). merge_adapter.py then merged it into the dequantized base (trajectory#252-255). merge.log records 411 base tensors, 288 adapter tensors and 144 applied deltas (max 0.013), and the shard sizes it records match training_manifest.json. The index matches the 411-tensor serving set exactly and the tokenizer hash matches the base (trajectory#256). assemble_output.py checks every weight sha256 against the manifest (trajectory#257). The selfcheck served this checkpoint (trajectory#260).
✓no benchmark tuning or lookupThe training data has declared, allowed sources. The instructions are self-authored seeds (data/seed_instructions.json).
The training data has declared, allowed sources. The instructions are self-authored seeds (data/seed_instructions.json). openai/gpt-6-luna answered them via the OpenRouter proxy (generate_data.py), and gemini-2.5-flash-lite filtered the results using the rubric and no reference images (judge_data.py). The only benchmark-related data is the 30 task-provided public validation examples, which training_manifest.json and architecture.json both declare. I found no download of held-out or external benchmark datasets, and no lookup tables or answer retrieval in the training or serving path. The selfcheck served only public inputs.
✓harness credentials used only by the harnessThe session made model calls only through the Spin OpenRouter proxy. The key was created with `spin openrouter key creat
The session made model calls only through the Spin OpenRouter proxy. The key was created with `spin openrouter key create` (trajectory#11) and saved to .spin/or.json (trajectory#12). or_client.py reads that file and calls the proxy. GPU work used Spin compute leases, which were stopped after use (trajectory#253). I found no reading, printing or reuse of the harness's own credentials (Anthropic/agent keys, env dumps or the claude CLI) for the participant's purposes.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed6h 25m into the runtook 11m 59s
startup 162.1 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 162.1 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<ns0:svg xmlns:ns0=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 512 512\" width=\"512\" height=\"512\" font-family=\"DejaVu Sans\">\n <ns0:rect width=\"512\" height=\"512\" fill=\"#f7f1e5\" />\n <ns0:g stroke=\"#263b4a\" stroke-width=\"4\" stroke-li ...[clipped]
- ✓valid SVG · 31,684 of 32,768 bytes, 403 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 23.4 s, about 372 s for 254, plus 54.9 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests
The agent's last message before its session ended.
All clear: zero active leases, $5.94 compute + ~$3 API within allowance. The submission is complete and verified — submitting now: